Jul 12 17:51:40.893768 ip-10-0-131-122 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 12 17:51:40.893780 ip-10-0-131-122 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 12 17:51:40.893787 ip-10-0-131-122 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 12 17:51:40.894057 ip-10-0-131-122 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 12 17:51:50.958664 ip-10-0-131-122 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 12 17:51:50.958686 ip-10-0-131-122 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 87ab5d93171d41b0ab94cbde4c79da3d --
Jul 12 17:53:44.681766 ip-10-0-131-122 systemd[1]: Starting Kubernetes Kubelet...
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.059606    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065705    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065720    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065724    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065727    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065730    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065733    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065736    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 17:53:45.065781 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065739    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065741    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065744    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065746    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065749    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065752    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065756    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065760    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065763    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065766    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065769    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065772    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065775    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065777    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065780    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065783    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065785    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065788    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065791    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065794    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 17:53:45.080344 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065796    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065799    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065803    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065805    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065808    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065811    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065813    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065816    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065820    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065823    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065826    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065829    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065831    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065834    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065836    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065839    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065841    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065844    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065846    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065849    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 17:53:45.082077 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065851    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065854    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065857    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065861    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065864    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065866    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065869    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065871    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065873    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065876    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065879    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065881    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065884    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065886    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065889    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065891    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065894    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065896    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065899    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065901    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 17:53:45.084310 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065904    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065907    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065909    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065912    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065916    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065919    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065922    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065924    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065927    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065929    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065932    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065935    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065937    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065940    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065942    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065945    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065948    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065951    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065954    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065956    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 17:53:45.095999 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065959    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065962    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065965    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065967    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065970    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065972    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065975    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065978    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065980    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065983    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065985    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.065988    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066673    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066680    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066683    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066685    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066688    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066691    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066693    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066696    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 17:53:45.098102 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066699    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066701    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066704    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066707    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066709    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066712    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066714    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066717    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066719    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066723    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066726    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066729    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066732    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066734    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066737    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066740    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066742    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066745    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066747    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066750    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 17:53:45.099885 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066752    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066755    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066757    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066761    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066765    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066769    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066772    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066775    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066787    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066790    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066793    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066796    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066799    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066802    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066805    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066807    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066810    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066812    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066815    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066817    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 17:53:45.112541 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066820    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066823    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066826    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066829    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066832    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066834    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066837    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066840    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066842    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066845    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066847    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066850    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066853    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066855    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066857    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066860    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066862    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066867    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066871    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066873    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 17:53:45.114253 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066876    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066878    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066880    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066883    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066886    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066891    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066894    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066896    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066899    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066901    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066904    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066907    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066909    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066912    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066915    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066919    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066922    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066925    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066927    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 17:53:45.116357 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066930    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066932    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066935    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066937    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066940    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066942    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066944    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066947    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066949    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066952    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066955    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.066957    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067320    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067325    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067327    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067330    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067332    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067335    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067340    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067343    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 17:53:45.128109 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067346    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067348    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067351    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067353    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067356    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067359    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067362    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067364    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067367    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067370    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067373    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067375    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067378    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067380    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067382    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067385    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067387    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067390    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067392    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067395    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 17:53:45.130795 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067397    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 17:53:45.129389 ip-10-0-131-122 systemd[1]: Started Kubernetes Kubelet.
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067399    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067402    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067404    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067407    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067410    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067413    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067415    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067417    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067420    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067424    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067426    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067429    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067431    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067434    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067436    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067439    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067441    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067444    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067446    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 17:53:45.203427 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067449    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067452    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067454    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067456    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067459    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067462    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067464    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067467    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067469    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067472    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067474    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067478    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067481    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067484    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067488    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067491    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067494    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067496    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067499    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067502    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 17:53:45.368013 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067505    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067508    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067511    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067529    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067532    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067535    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067537    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067540    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067542    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067545    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067547    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067550    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067552    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067555    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067558    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067561    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067564    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067566    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067568    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 17:53:45.381650 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067571    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067573    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067576    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067578    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067581    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067584    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067587    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067592    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067596    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067598    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067601    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.067604    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067672    2578 flags.go:64] FLAG: --address="0.0.0.0"
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067679    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067684    2578 flags.go:64] FLAG: --anonymous-auth="true"
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067690    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067694    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067697    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067700    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067704    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067707    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 12 17:53:45.383969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067710    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067713    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067717    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067719    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067722    2578 flags.go:64] FLAG: --cgroup-root=""
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067725    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067728    2578 flags.go:64] FLAG: --client-ca-file=""
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067731    2578 flags.go:64] FLAG: --cloud-provider="external"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067734    2578 flags.go:64] FLAG: --cluster-dns="[]"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067738    2578 flags.go:64] FLAG: --cluster-domain=""
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067741    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067743    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067747    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067750    2578 flags.go:64] FLAG: --container-log-max-files="5"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067753    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067756    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067759    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067762    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067766    2578 flags.go:64] FLAG: --contention-profiling="false"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067769    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067772    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067775    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067777    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067781    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 12 17:53:45.385332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067784    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067787    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067791    2578 flags.go:64] FLAG: --enable-load-reader="false"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067794    2578 flags.go:64] FLAG: --enable-server="true"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067796    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067804    2578 flags.go:64] FLAG: --event-burst="100"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067807    2578 flags.go:64] FLAG: --event-qps="50"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067809    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067812    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067815    2578 flags.go:64] FLAG: --eviction-hard=""
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067823    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067826    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067828    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067831    2578 flags.go:64] FLAG: --eviction-soft=""
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067834    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067837    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067840    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067843    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067845    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067848    2578 flags.go:64] FLAG: --fail-swap-on="true"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067851    2578 flags.go:64] FLAG: --feature-gates=""
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067854    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067857    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067860    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067863    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067866    2578 flags.go:64] FLAG: --healthz-port="10248"
Jul 12 17:53:45.387008 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067870    2578 flags.go:64] FLAG: --help="false"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067873    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067876    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067879    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067881    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067885    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067888    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067891    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067895    2578 flags.go:64] FLAG: --image-service-endpoint=""
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067898    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067900    2578 flags.go:64] FLAG: --kube-api-burst="100"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067903    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067906    2578 flags.go:64] FLAG: --kube-api-qps="50"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067909    2578 flags.go:64] FLAG: --kube-reserved=""
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067912    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067914    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067917    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067920    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067922    2578 flags.go:64] FLAG: --lock-file=""
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067925    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067928    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067931    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067935    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 12 17:53:45.388100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067939    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067942    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067944    2578 flags.go:64] FLAG: --logging-format="text"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067947    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067950    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067953    2578 flags.go:64] FLAG: --manifest-url=""
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067955    2578 flags.go:64] FLAG: --manifest-url-header=""
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067959    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067962    2578 flags.go:64] FLAG: --max-open-files="1000000"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067967    2578 flags.go:64] FLAG: --max-pods="110"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067970    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067972    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067975    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067978    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067981    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067984    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067986    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067994    2578 flags.go:64] FLAG: --node-status-max-images="50"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.067997    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068000    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068003    2578 flags.go:64] FLAG: --pod-cidr=""
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068005    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068010    2578 flags.go:64] FLAG: --pod-manifest-path=""
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068013    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 12 17:53:45.389195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068016    2578 flags.go:64] FLAG: --pods-per-core="0"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068018    2578 flags.go:64] FLAG: --port="10250"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068021    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068024    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c797d547725df767"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068027    2578 flags.go:64] FLAG: --qos-reserved=""
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068030    2578 flags.go:64] FLAG: --read-only-port="10255"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068032    2578 flags.go:64] FLAG: --register-node="true"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068035    2578 flags.go:64] FLAG: --register-with-taints=""
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068038    2578 flags.go:64] FLAG: --registry-burst="10"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068042    2578 flags.go:64] FLAG: --registry-qps="5"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068044    2578 flags.go:64] FLAG: --reserved-cpus=""
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068047    2578 flags.go:64] FLAG: --reserved-memory=""
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068050    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068053    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068056    2578 flags.go:64] FLAG: --rotate-certificates="false"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068059    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068061    2578 flags.go:64] FLAG: --runonce="false"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068065    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068069    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068072    2578 flags.go:64] FLAG: --seccomp-default="false"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068074    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068077    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068080    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068083    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068086    2578 flags.go:64] FLAG: --storage-driver-password="root"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068089    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 12 17:53:45.390246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068094    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068097    2578 flags.go:64] FLAG: --storage-driver-user="root"
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068099    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068102    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068105    2578 flags.go:64] FLAG: --system-cgroups=""
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068108    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068113    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068115    2578 flags.go:64] FLAG: --tls-cert-file=""
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068118    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068121    2578 flags.go:64] FLAG: --tls-min-version=""
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068124    2578 flags.go:64] FLAG: --tls-private-key-file=""
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068127    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068130    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068132    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068135    2578 flags.go:64] FLAG: --v="2"
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068139    2578 flags.go:64] FLAG: --version="false"
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068143    2578 flags.go:64] FLAG: --vmodule=""
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068146    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.068149    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068258    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068261    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068265    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068269    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 17:53:45.391618 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068273    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068276    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068278    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068281    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068284    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068286    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068289    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068291    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068294    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068297    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068300    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068302    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068305    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068307    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068310    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068312    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068315    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068317    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068320    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 17:53:45.392818 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068322    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068325    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068328    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068330    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068333    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068335    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068338    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068340    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068343    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068345    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068348    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068350    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068352    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068356    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068359    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068361    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068363    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068366    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068369    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 17:53:45.393674 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068371    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068374    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068378    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068383    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068385    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068388    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068391    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068394    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068396    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068399    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068402    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068404    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068407    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068409    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068412    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068415    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068419    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068422    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068425    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 17:53:45.394479 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068427    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068429    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068433    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068435    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068437    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068440    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068442    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068446    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068448    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068451    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068453    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068456    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068458    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068461    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068463    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068465    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068469    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068472    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068474    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068477    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 17:53:45.395340 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068479    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068482    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068484    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068487    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068489    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068492    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068494    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068497    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068499    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068502    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068504    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068507    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068509    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068512    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068529    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068533    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068536    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 17:53:45.396193 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.068538    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.069231    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.075921    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.075937    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.075955    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.075961    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076023    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076029    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076032    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076035    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076038    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076040    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076043    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076045    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 17:53:45.397033 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076048    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076050    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076053    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076056    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076060    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076063    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076066    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076069    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076072    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076075    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076078    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076081    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076084    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076087    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076090    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076093    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076096    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076098    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076101    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 17:53:45.397668 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076104    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076106    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076109    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076114    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076117    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076120    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076124    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076127    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076130    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076133    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076137    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076140    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076143    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076145    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076148    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076151    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076153    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076156    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076158    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076161    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 17:53:45.398535 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076164    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076166    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076168    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076171    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076174    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076176    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076179    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076181    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076184    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076186    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076189    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076192    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076194    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076198    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076201    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076205    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076208    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076210    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076213    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076215    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 17:53:45.399372 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076218    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076220    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076224    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076226    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076229    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076232    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076234    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076237    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076240    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076242    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076244    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076247    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076249    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076252    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076254    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076256    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076259    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076262    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076264    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076266    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 17:53:45.400236 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076269    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076271    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076274    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076277    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076279    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076281    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076284    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076287    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076290    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076292    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076295    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076297    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.076302    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076446    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076451    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 12 17:53:45.401090 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076456    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076458    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076461    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076464    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076466    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076469    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076471    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076474    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076477    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076479    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076482    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076484    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076486    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076489    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076491    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076494    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076497    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076499    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076501    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 12 17:53:45.401730 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076504    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076506    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076509    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076512    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076529    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076534    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076538    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076541    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076543    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076546    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076548    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076551    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076553    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076557    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076560    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076562    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076565    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076567    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076571    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 12 17:53:45.404710 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076574    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076578    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076581    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076585    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076588    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076590    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076593    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076595    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076599    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076602    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076605    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076608    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076610    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076612    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076615    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076617    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076620    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076623    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076627    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 12 17:53:45.405794 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076629    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076632    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076634    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076637    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076639    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076641    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076644    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076648    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076650    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076653    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076655    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076658    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076661    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076663    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076666    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076668    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076671    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076673    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076676    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076678    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 12 17:53:45.586117 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076681    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076683    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076685    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076688    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076690    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076693    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076695    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076698    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076700    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076703    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076706    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076709    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076712    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076714    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076717    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076719    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076722    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076724    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076727    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 12 17:53:45.587055 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:45.076731    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 12 17:53:45.587655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.076735    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 12 17:53:45.587655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.076851    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 12 17:53:45.587655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.079173    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 12 17:53:45.587655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.080613    2578 server.go:1019] "Starting client certificate rotation"
Jul 12 17:53:45.587655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.080716    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 12 17:53:45.587655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.080750    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 12 17:53:45.587655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.083337    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 12 17:53:45.587655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.085070    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 12 17:53:45.587655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.096768    2578 log.go:25] "Validated CRI v1 runtime API"
Jul 12 17:53:45.587655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.100140    2578 log.go:25] "Validated CRI v1 image API"
Jul 12 17:53:45.587655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.101110    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 12 17:53:45.587655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.103826    2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a6362cf0-4c5d-4e40-ab53-597bf2220f19:/dev/nvme0n1p3 d197eed2-6dd3-4ce3-b92d-680c96b5b41e:/dev/nvme0n1p4]
Jul 12 17:53:45.588061 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.103839    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 12 17:53:45.588135 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.108545    2578 manager.go:216] Machine: {Timestamp:2026-07-12 17:53:45.107317173 +0000 UTC m=+0.307436818 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3199199 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fd807c191c8dc3b06e546c91b4c51 SystemUUID:ec2fd807-c191-c8dc-3b06-e546c91b4c51 BootID:87ab5d93-171d-41b0-ab94-cbde4c79da3d Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:57:81:1a:cf:bf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:57:81:1a:cf:bf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:ab:a2:dd:b3:95 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 12 17:53:45.588135 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.108641    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 12 17:53:45.588135 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.108705    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 12 17:53:45.588135 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.110084    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 12 17:53:45.588135 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.111832    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 12 17:53:45.588135 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.111851    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-122.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 12 17:53:45.588135 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.113000    2578 topology_manager.go:138] "Creating topology manager with none policy"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.113010    2578 container_manager_linux.go:309] "Creating device plugin manager"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.113018    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.113539    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.115213    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.118590    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.118713    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.118739    2578 kubelet.go:486] "Attempting to sync node with API server"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.118752    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.118762    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.118770    2578 kubelet.go:398] "Adding apiserver pod source"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.118786    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.119200    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.119227    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.121237    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.123984    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.124010    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.124162    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.124173    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.124179    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.124185    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.124193    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.124198    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.124566    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.124574    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.124580    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.124585    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.124605    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.124616    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.125370    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.125377    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.128678    2578 server.go:1268] "Started kubelet"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.128766    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.128827    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.128892    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.132915    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.133477    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-122.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.134091    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-122.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.134199    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jhrrm"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.134389    2578 server.go:310] "Adding debug handlers to kubelet server"
Jul 12 17:53:45.588436 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.134971    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.135120    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.138206    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-122.ec2.internal.18c19c570e22543d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-122.ec2.internal,UID:ip-10-0-131-122.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-122.ec2.internal,},FirstTimestamp:2026-07-12 17:53:45.128645693 +0000 UTC m=+0.328765339,LastTimestamp:2026-07-12 17:53:45.128645693 +0000 UTC m=+0.328765339,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-122.ec2.internal,}"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.141267    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jhrrm"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.141633    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.141637    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.141738    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.142321    2578 factory.go:55] Registering systemd factory
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.142362    2578 factory.go:223] Registration of the systemd container factory successfully
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.142323    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.142325    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.142402    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.142507    2578 reconstruct.go:97] "Volume reconstruction finished"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.142530    2578 reconciler.go:29] "Reconciler: start to sync state"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.142597    2578 factory.go:153] Registering CRI-O factory
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.142607    2578 factory.go:223] Registration of the crio container factory successfully
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.142648    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.142666    2578 factory.go:103] Registering Raw factory
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.142676    2578 manager.go:1196] Started watching for new ooms in manager
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.142993    2578 manager.go:349] Starting recovery of all containers
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.144602    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.144812    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-122.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.149549    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.152365    2578 manager.go:354] Recovery completed
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.156375    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.158082    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.158104    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.158114    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientPID"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.158496    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.158505    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.158535    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.158505    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.159598    2578 policy_none.go:49] "None policy: Start"
Jul 12 17:53:45.589539 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.159612    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.159624    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.160588    2578 policy_none.go:47] "Start"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.161268    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.161294    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.161305    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientPID"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.226621    2578 manager.go:337] "Starting Device Plugin manager"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.226681    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.226693    2578 server.go:87] "Starting device plugin registration server"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.226915    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.226926    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.227045    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.227115    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.227123    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.227688    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.227739    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.318903    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.318938    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.318956    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.318988    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.320691    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.327434    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.328171    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.328220    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.328233    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientPID"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.328259    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.335659    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.335676    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-122.ec2.internal\": node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.346472    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.419256    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-122.ec2.internal"]
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.419427    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.420969    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.420994    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.421007    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientPID"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.422103    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.422200    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.422238    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 17:53:45.590490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.422718    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.422725    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.422744    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.422757    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientPID"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.422745    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.422838    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientPID"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.425841    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.425862    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.426568    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientMemory"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.426591    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasNoDiskPressure"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.426600    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeHasSufficientPID"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.443655    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7b4433a31c1223ad0a8204daf58a4cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal\" (UID: \"b7b4433a31c1223ad0a8204daf58a4cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.443680    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7b4433a31c1223ad0a8204daf58a4cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal\" (UID: \"b7b4433a31c1223ad0a8204daf58a4cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.443702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9235ba932a6066a0b11ef970f30044e4-config\") pod \"kube-apiserver-proxy-ip-10-0-131-122.ec2.internal\" (UID: \"9235ba932a6066a0b11ef970f30044e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.446739    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.457991    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-122.ec2.internal\" not found" node="ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.462205    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-122.ec2.internal\" not found" node="ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.544314    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7b4433a31c1223ad0a8204daf58a4cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal\" (UID: \"b7b4433a31c1223ad0a8204daf58a4cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.544336    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7b4433a31c1223ad0a8204daf58a4cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal\" (UID: \"b7b4433a31c1223ad0a8204daf58a4cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.544385    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7b4433a31c1223ad0a8204daf58a4cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal\" (UID: \"b7b4433a31c1223ad0a8204daf58a4cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.591499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.544412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9235ba932a6066a0b11ef970f30044e4-config\") pod \"kube-apiserver-proxy-ip-10-0-131-122.ec2.internal\" (UID: \"9235ba932a6066a0b11ef970f30044e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.592126 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.544425    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7b4433a31c1223ad0a8204daf58a4cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal\" (UID: \"b7b4433a31c1223ad0a8204daf58a4cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.592126 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.544451    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9235ba932a6066a0b11ef970f30044e4-config\") pod \"kube-apiserver-proxy-ip-10-0-131-122.ec2.internal\" (UID: \"9235ba932a6066a0b11ef970f30044e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.592126 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.547710    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:45.771202 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.648541    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:45.771202 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.749459    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:45.771202 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.760390    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.771202 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.766807    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:45.893448 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.849688    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:46.017703 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:45.950764    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:46.017703 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:45.978075    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 17:53:46.147714 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:46.051862    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:46.147714 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:46.077934    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 17:53:46.147714 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:46.080214    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 12 17:53:46.147714 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:46.080327    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 12 17:53:46.147714 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:46.080356    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 12 17:53:46.147714 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:46.080380    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 12 17:53:46.147714 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:46.145017    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 17:48:45 +0000 UTC" deadline="2028-02-05 02:54:01.1823035 +0000 UTC"
Jul 12 17:53:46.147714 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:46.145050    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13737h0m15.037256194s"
Jul 12 17:53:46.153809 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:46.152632    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:46.266276 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:46.252841    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:46.389206 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:46.353601    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-122.ec2.internal\" not found"
Jul 12 17:53:46.389206 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:46.376299    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 17:53:46.512808 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:46.442417    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:46.512808 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:46.454976    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 12 17:53:46.512808 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:46.456055    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-122.ec2.internal"
Jul 12 17:53:46.512808 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:46.462550    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.120699    2578 apiserver.go:52] "Watching apiserver"
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.131294    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.131663    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-fplg4","openshift-network-diagnostics/network-check-target-ckkjw","openshift-ovn-kubernetes/ovnkube-node-trbvl","kube-system/konnectivity-agent-jrpgp","kube-system/kube-apiserver-proxy-ip-10-0-131-122.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4","openshift-image-registry/node-ca-b2njf","openshift-multus/network-metrics-daemon-k4llc","openshift-network-operator/iptables-alerter-zgbc7","kube-system/global-pull-secret-syncer-97vpp","openshift-cluster-node-tuning-operator/tuned-vvzzz","openshift-dns/node-resolver-89mdh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal","openshift-multus/multus-8wsqz"]
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.133159    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b2njf"
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.134267    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.134357    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.135495    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.136621    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jrpgp"
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.137731    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.137817    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.137853    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w52gl\""
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.137873    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.137943    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.137977    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.138188    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.138269    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zfktj\""
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.139262    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.140052    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.140070    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.140082    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.140124    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.140130    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.140444    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zgbc7"
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.140575    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.141787    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.142666    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 12 17:53:47.157481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.142959    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.143000    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.143661    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rf5jb\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.144229    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.144349    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.144423    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.144463    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.144492    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.144540    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.144557    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.144748    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.144807    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.144907    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.145478    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.145491    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qw8kl\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.145515    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.145544    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lcbzt\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.145569    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.145482    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jhdfn\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.145521    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9fd42\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.145496    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.146337    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.149463    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-89mdh"
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.149522    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.149761    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tvv4p\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.150306    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 12 17:53:47.159165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151405    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-run-netns\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.163959 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151433    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.163959 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151455    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-cnibin\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.163959 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151474    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-run-netns\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.163959 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151506    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b092918b-d322-40b1-b850-33b856bb1ba4-host\") pod \"node-ca-b2njf\" (UID: \"b092918b-d322-40b1-b850-33b856bb1ba4\") " pod="openshift-image-registry/node-ca-b2njf"
Jul 12 17:53:47.163959 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151541    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8vt8\" (UniqueName: \"kubernetes.io/projected/b092918b-d322-40b1-b850-33b856bb1ba4-kube-api-access-t8vt8\") pod \"node-ca-b2njf\" (UID: \"b092918b-d322-40b1-b850-33b856bb1ba4\") " pod="openshift-image-registry/node-ca-b2njf"
Jul 12 17:53:47.163959 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151569    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-env-overrides\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.163959 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151593    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ca436ac-91ae-4edb-975c-98ee1847a28a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.163959 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151614    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-systemd-units\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.163959 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151633    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-socket-dir\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.163959 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151654    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-device-dir\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.163959 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151675    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qknnv\" (UniqueName: \"kubernetes.io/projected/919b372b-2a09-4b02-a63b-79c46b645580-kube-api-access-qknnv\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.163959 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151704    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-run-k8s-cni-cncf-io\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.163959 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151736    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-var-lib-cni-bin\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.164397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151769    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-slash\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.164397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-run-systemd\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.164397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151818    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-multus-socket-dir-parent\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.164397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151834    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61e71b08-89e9-4f55-a6db-a5f0dcd10f2a-konnectivity-ca\") pod \"konnectivity-agent-jrpgp\" (UID: \"61e71b08-89e9-4f55-a6db-a5f0dcd10f2a\") " pod="kube-system/konnectivity-agent-jrpgp"
Jul 12 17:53:47.164397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151848    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-cni-netd\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.164397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151866    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-multus-conf-dir\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.164397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151888    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-etc-kubernetes\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.164397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151909    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ca436ac-91ae-4edb-975c-98ee1847a28a-system-cni-dir\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.164397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151923    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ca436ac-91ae-4edb-975c-98ee1847a28a-cni-binary-copy\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.164397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151938    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ca436ac-91ae-4edb-975c-98ee1847a28a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.164397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151956    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.164397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.151980    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-var-lib-cni-multus\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.164397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152007    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-multus-daemon-config\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152034    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ca436ac-91ae-4edb-975c-98ee1847a28a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152057    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9d79\" (UniqueName: \"kubernetes.io/projected/4411fcad-827e-4fe9-9165-8eaba1e75226-kube-api-access-w9d79\") pod \"network-metrics-daemon-k4llc\" (UID: \"4411fcad-827e-4fe9-9165-8eaba1e75226\") " pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152073    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mmxc\" (UniqueName: \"kubernetes.io/projected/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-kube-api-access-2mmxc\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152090    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-hostroot\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152104    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-run-multus-certs\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152126    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-cni-bin\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152146    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-system-cni-dir\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152160    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-888bx\" (UniqueName: \"kubernetes.io/projected/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-kube-api-access-888bx\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152175    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-sys-fs\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b092918b-d322-40b1-b850-33b856bb1ba4-serviceca\") pod \"node-ca-b2njf\" (UID: \"b092918b-d322-40b1-b850-33b856bb1ba4\") " pod="openshift-image-registry/node-ca-b2njf"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152224    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61e71b08-89e9-4f55-a6db-a5f0dcd10f2a-agent-certs\") pod \"konnectivity-agent-jrpgp\" (UID: \"61e71b08-89e9-4f55-a6db-a5f0dcd10f2a\") " pod="kube-system/konnectivity-agent-jrpgp"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152238    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-ovnkube-config\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-registration-dir\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.165262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152263    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 12 17:53:47.167389 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152282    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-etc-selinux\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.167389 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152306    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztznj\" (UniqueName: \"kubernetes.io/projected/9ca436ac-91ae-4edb-975c-98ee1847a28a-kube-api-access-ztznj\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.167389 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152328    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-run-openvswitch\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.167389 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-run-ovn\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.167389 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152382    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-ovn-node-metrics-cert\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.167389 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152396    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ca436ac-91ae-4edb-975c-98ee1847a28a-cnibin\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.167389 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152413    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a01ca45-b0df-41d8-9262-6a6f97d4a6ba-host-slash\") pod \"iptables-alerter-zgbc7\" (UID: \"8a01ca45-b0df-41d8-9262-6a6f97d4a6ba\") " pod="openshift-network-operator/iptables-alerter-zgbc7"
Jul 12 17:53:47.167389 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152430    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-var-lib-openvswitch\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.167389 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152478    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-cni-binary-copy\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.167389 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152494    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-var-lib-kubelet\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.167389 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152548    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgms6\" (UniqueName: \"kubernetes.io/projected/8a01ca45-b0df-41d8-9262-6a6f97d4a6ba-kube-api-access-vgms6\") pod \"iptables-alerter-zgbc7\" (UID: \"8a01ca45-b0df-41d8-9262-6a6f97d4a6ba\") " pod="openshift-network-operator/iptables-alerter-zgbc7"
Jul 12 17:53:47.167389 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152574    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs\") pod \"network-metrics-daemon-k4llc\" (UID: \"4411fcad-827e-4fe9-9165-8eaba1e75226\") " pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:47.167389 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152596    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-kubelet\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.167836 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152630    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-run-ovn-kubernetes\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.167836 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152662    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-etc-openvswitch\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.167836 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152689    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-node-log\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.167836 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152690    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 12 17:53:47.167836 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152707    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-ovnkube-script-lib\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.167836 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152730    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-multus-cni-dir\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.167836 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152736    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-58ljb\""
Jul 12 17:53:47.167836 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152749    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ca436ac-91ae-4edb-975c-98ee1847a28a-os-release\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.167836 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152768    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a01ca45-b0df-41d8-9262-6a6f97d4a6ba-iptables-alerter-script\") pod \"iptables-alerter-zgbc7\" (UID: \"8a01ca45-b0df-41d8-9262-6a6f97d4a6ba\") " pod="openshift-network-operator/iptables-alerter-zgbc7"
Jul 12 17:53:47.167836 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152782    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-log-socket\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.167836 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152797    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-os-release\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.167836 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.152833    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx29b\" (UniqueName: \"kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b\") pod \"network-check-target-ckkjw\" (UID: \"b4296c55-a6e8-4c34-93b6-3e72514d30bc\") " pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:53:47.167836 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.158557    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.186823    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-27xrz"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.196584    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-27xrz"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.243364    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253462    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-etc-openvswitch\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253488    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-node-log\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253509    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-ovnkube-script-lib\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253540    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-multus-cni-dir\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253561    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ca436ac-91ae-4edb-975c-98ee1847a28a-os-release\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253571    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-etc-openvswitch\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a01ca45-b0df-41d8-9262-6a6f97d4a6ba-iptables-alerter-script\") pod \"iptables-alerter-zgbc7\" (UID: \"8a01ca45-b0df-41d8-9262-6a6f97d4a6ba\") " pod="openshift-network-operator/iptables-alerter-zgbc7"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253581    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-node-log\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253604    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-log-socket\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253652    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-os-release\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253680    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx29b\" (UniqueName: \"kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b\") pod \"network-check-target-ckkjw\" (UID: \"b4296c55-a6e8-4c34-93b6-3e72514d30bc\") " pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253734    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-run-netns\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.280268 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253748    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ca436ac-91ae-4edb-975c-98ee1847a28a-os-release\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253762    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253767    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-log-socket\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253767    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-multus-cni-dir\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253791    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-sysctl-d\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253813    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f41ef213-692e-4bc5-99f0-7dca93e13104-tmp\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253819    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-run-netns\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-cnibin\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253838    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-os-release\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253846    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-run-netns\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253870    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253879    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-run-netns\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253883    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b092918b-d322-40b1-b850-33b856bb1ba4-host\") pod \"node-ca-b2njf\" (UID: \"b092918b-d322-40b1-b850-33b856bb1ba4\") " pod="openshift-image-registry/node-ca-b2njf"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253910    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8vt8\" (UniqueName: \"kubernetes.io/projected/b092918b-d322-40b1-b850-33b856bb1ba4-kube-api-access-t8vt8\") pod \"node-ca-b2njf\" (UID: \"b092918b-d322-40b1-b850-33b856bb1ba4\") " pod="openshift-image-registry/node-ca-b2njf"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-cnibin\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.291611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253937    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-env-overrides\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.253937    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b092918b-d322-40b1-b850-33b856bb1ba4-host\") pod \"node-ca-b2njf\" (UID: \"b092918b-d322-40b1-b850-33b856bb1ba4\") " pod="openshift-image-registry/node-ca-b2njf"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254048    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-kubernetes\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254071    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-systemd\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254089    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ca436ac-91ae-4edb-975c-98ee1847a28a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254106    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-systemd-units\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254125    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-socket-dir\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254148    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-device-dir\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254175    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-systemd-units\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254174    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qknnv\" (UniqueName: \"kubernetes.io/projected/919b372b-2a09-4b02-a63b-79c46b645580-kube-api-access-qknnv\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254220    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-run-k8s-cni-cncf-io\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254246    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-device-dir\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254247    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-var-lib-cni-bin\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-slash\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.397933 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254279    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-socket-dir\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254288    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-run-k8s-cni-cncf-io\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254325    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-var-lib-cni-bin\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-slash\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254354    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-run-systemd\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254379    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-multus-socket-dir-parent\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61e71b08-89e9-4f55-a6db-a5f0dcd10f2a-konnectivity-ca\") pod \"konnectivity-agent-jrpgp\" (UID: \"61e71b08-89e9-4f55-a6db-a5f0dcd10f2a\") " pod="kube-system/konnectivity-agent-jrpgp"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-cni-netd\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254453    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-run-systemd\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254467    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-multus-conf-dir\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254485    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-etc-kubernetes\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254505    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-sysctl-conf\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254513    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-multus-conf-dir\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254525    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ca436ac-91ae-4edb-975c-98ee1847a28a-system-cni-dir\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254537    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-cni-netd\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.398737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254485    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-multus-socket-dir-parent\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254558    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzbhq\" (UniqueName: \"kubernetes.io/projected/734454b5-e121-4543-ab79-ecfe0d985f96-kube-api-access-lzbhq\") pod \"node-resolver-89mdh\" (UID: \"734454b5-e121-4543-ab79-ecfe0d985f96\") " pod="openshift-dns/node-resolver-89mdh"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254572    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-etc-kubernetes\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254577    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-tuned\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254589    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ca436ac-91ae-4edb-975c-98ee1847a28a-system-cni-dir\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254605    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ca436ac-91ae-4edb-975c-98ee1847a28a-cni-binary-copy\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254629    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ca436ac-91ae-4edb-975c-98ee1847a28a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254649    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254671    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-var-lib-cni-multus\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254692    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-multus-daemon-config\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254714    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ca436ac-91ae-4edb-975c-98ee1847a28a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9d79\" (UniqueName: \"kubernetes.io/projected/4411fcad-827e-4fe9-9165-8eaba1e75226-kube-api-access-w9d79\") pod \"network-metrics-daemon-k4llc\" (UID: \"4411fcad-827e-4fe9-9165-8eaba1e75226\") " pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254770    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254773    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mmxc\" (UniqueName: \"kubernetes.io/projected/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-kube-api-access-2mmxc\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.400437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254796    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-hostroot\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254818    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-run-multus-certs\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254859    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-modprobe-d\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254880    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-cni-bin\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254908    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-system-cni-dir\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254916    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-cni-bin\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254929    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-888bx\" (UniqueName: \"kubernetes.io/projected/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-kube-api-access-888bx\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254954    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-sysconfig\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254985    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-sys-fs\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255008    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b092918b-d322-40b1-b850-33b856bb1ba4-serviceca\") pod \"node-ca-b2njf\" (UID: \"b092918b-d322-40b1-b850-33b856bb1ba4\") " pod="openshift-image-registry/node-ca-b2njf"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255031    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61e71b08-89e9-4f55-a6db-a5f0dcd10f2a-agent-certs\") pod \"konnectivity-agent-jrpgp\" (UID: \"61e71b08-89e9-4f55-a6db-a5f0dcd10f2a\") " pod="kube-system/konnectivity-agent-jrpgp"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255056    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-ovnkube-config\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255082    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-registration-dir\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255112    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-hostroot\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255118    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-etc-selinux\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.400877 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255147    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-sys\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-lib-modules\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255199    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztznj\" (UniqueName: \"kubernetes.io/projected/9ca436ac-91ae-4edb-975c-98ee1847a28a-kube-api-access-ztznj\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-run-openvswitch\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255235    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-run-ovn\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255242    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-registration-dir\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-ovn-node-metrics-cert\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255289    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/734454b5-e121-4543-ab79-ecfe0d985f96-hosts-file\") pod \"node-resolver-89mdh\" (UID: \"734454b5-e121-4543-ab79-ecfe0d985f96\") " pod="openshift-dns/node-resolver-89mdh"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255315    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/734454b5-e121-4543-ab79-ecfe0d985f96-tmp-dir\") pod \"node-resolver-89mdh\" (UID: \"734454b5-e121-4543-ab79-ecfe0d985f96\") " pod="openshift-dns/node-resolver-89mdh"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255342    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-kubelet-config\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255359    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-etc-selinux\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255367    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-dbus\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.254740    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-var-lib-cni-multus\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255443    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/919b372b-2a09-4b02-a63b-79c46b645580-sys-fs\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.402177 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255084    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-run-multus-certs\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255715    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-run-openvswitch\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255933    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-run-ovn\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255976    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-system-cni-dir\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.255976    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ca436ac-91ae-4edb-975c-98ee1847a28a-cnibin\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256013    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ca436ac-91ae-4edb-975c-98ee1847a28a-cnibin\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a01ca45-b0df-41d8-9262-6a6f97d4a6ba-host-slash\") pod \"iptables-alerter-zgbc7\" (UID: \"8a01ca45-b0df-41d8-9262-6a6f97d4a6ba\") " pod="openshift-network-operator/iptables-alerter-zgbc7"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-var-lib-openvswitch\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256066    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-cni-binary-copy\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256076    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a01ca45-b0df-41d8-9262-6a6f97d4a6ba-host-slash\") pod \"iptables-alerter-zgbc7\" (UID: \"8a01ca45-b0df-41d8-9262-6a6f97d4a6ba\") " pod="openshift-network-operator/iptables-alerter-zgbc7"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256081    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-var-lib-kubelet\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256101    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256118    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-var-lib-openvswitch\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256128    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-run\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256150    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-host-var-lib-kubelet\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.403047 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256156    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgms6\" (UniqueName: \"kubernetes.io/projected/8a01ca45-b0df-41d8-9262-6a6f97d4a6ba-kube-api-access-vgms6\") pod \"iptables-alerter-zgbc7\" (UID: \"8a01ca45-b0df-41d8-9262-6a6f97d4a6ba\") " pod="openshift-network-operator/iptables-alerter-zgbc7"
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256190    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs\") pod \"network-metrics-daemon-k4llc\" (UID: \"4411fcad-827e-4fe9-9165-8eaba1e75226\") " pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-kubelet\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256343    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-run-ovn-kubernetes\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-var-lib-kubelet\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.256387    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256402    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-run-ovn-kubernetes\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-host\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256449    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgdsz\" (UniqueName: \"kubernetes.io/projected/f41ef213-692e-4bc5-99f0-7dca93e13104-kube-api-access-zgdsz\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.256486    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs podName:4411fcad-827e-4fe9-9165-8eaba1e75226 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:47.756447485 +0000 UTC m=+2.956567118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs") pod "network-metrics-daemon-k4llc" (UID: "4411fcad-827e-4fe9-9165-8eaba1e75226") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256446    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-host-kubelet\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256726    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ca436ac-91ae-4edb-975c-98ee1847a28a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256903    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-env-overrides\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256995    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-ovnkube-config\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.404329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.256994    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-ovnkube-script-lib\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.257346    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.257830    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a01ca45-b0df-41d8-9262-6a6f97d4a6ba-iptables-alerter-script\") pod \"iptables-alerter-zgbc7\" (UID: \"8a01ca45-b0df-41d8-9262-6a6f97d4a6ba\") " pod="openshift-network-operator/iptables-alerter-zgbc7"
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.257970    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ca436ac-91ae-4edb-975c-98ee1847a28a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.258046    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b092918b-d322-40b1-b850-33b856bb1ba4-serviceca\") pod \"node-ca-b2njf\" (UID: \"b092918b-d322-40b1-b850-33b856bb1ba4\") " pod="openshift-image-registry/node-ca-b2njf"
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.258065    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ca436ac-91ae-4edb-975c-98ee1847a28a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.258125    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61e71b08-89e9-4f55-a6db-a5f0dcd10f2a-konnectivity-ca\") pod \"konnectivity-agent-jrpgp\" (UID: \"61e71b08-89e9-4f55-a6db-a5f0dcd10f2a\") " pod="kube-system/konnectivity-agent-jrpgp"
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.258626    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-multus-daemon-config\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.258638    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ca436ac-91ae-4edb-975c-98ee1847a28a-cni-binary-copy\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.259029    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-cni-binary-copy\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.262203    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-ovn-node-metrics-cert\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.262329    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61e71b08-89e9-4f55-a6db-a5f0dcd10f2a-agent-certs\") pod \"konnectivity-agent-jrpgp\" (UID: \"61e71b08-89e9-4f55-a6db-a5f0dcd10f2a\") " pod="kube-system/konnectivity-agent-jrpgp"
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.262562    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.262582    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 17:53:47.406458 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.262600    2578 projected.go:196] Error preparing data for projected volume kube-api-access-vx29b for pod openshift-network-diagnostics/network-check-target-ckkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:53:47.406896 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.262647    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b podName:b4296c55-a6e8-4c34-93b6-3e72514d30bc nodeName:}" failed. No retries permitted until 2026-07-12 17:53:47.762633868 +0000 UTC m=+2.962753504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vx29b" (UniqueName: "kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b") pod "network-check-target-ckkjw" (UID: "b4296c55-a6e8-4c34-93b6-3e72514d30bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:53:47.406896 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.264248    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8vt8\" (UniqueName: \"kubernetes.io/projected/b092918b-d322-40b1-b850-33b856bb1ba4-kube-api-access-t8vt8\") pod \"node-ca-b2njf\" (UID: \"b092918b-d322-40b1-b850-33b856bb1ba4\") " pod="openshift-image-registry/node-ca-b2njf"
Jul 12 17:53:47.406896 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.264711    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qknnv\" (UniqueName: \"kubernetes.io/projected/919b372b-2a09-4b02-a63b-79c46b645580-kube-api-access-qknnv\") pod \"aws-ebs-csi-driver-node-zjsl4\" (UID: \"919b372b-2a09-4b02-a63b-79c46b645580\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.406896 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.265532    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgms6\" (UniqueName: \"kubernetes.io/projected/8a01ca45-b0df-41d8-9262-6a6f97d4a6ba-kube-api-access-vgms6\") pod \"iptables-alerter-zgbc7\" (UID: \"8a01ca45-b0df-41d8-9262-6a6f97d4a6ba\") " pod="openshift-network-operator/iptables-alerter-zgbc7"
Jul 12 17:53:47.406896 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.265553    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-888bx\" (UniqueName: \"kubernetes.io/projected/98dd2ed4-4bfb-4e7b-acae-e90695a81f22-kube-api-access-888bx\") pod \"multus-8wsqz\" (UID: \"98dd2ed4-4bfb-4e7b-acae-e90695a81f22\") " pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.406896 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.265847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mmxc\" (UniqueName: \"kubernetes.io/projected/d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08-kube-api-access-2mmxc\") pod \"ovnkube-node-trbvl\" (UID: \"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08\") " pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.406896 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.266077    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztznj\" (UniqueName: \"kubernetes.io/projected/9ca436ac-91ae-4edb-975c-98ee1847a28a-kube-api-access-ztznj\") pod \"multus-additional-cni-plugins-fplg4\" (UID: \"9ca436ac-91ae-4edb-975c-98ee1847a28a\") " pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.406896 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.267316    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9d79\" (UniqueName: \"kubernetes.io/projected/4411fcad-827e-4fe9-9165-8eaba1e75226-kube-api-access-w9d79\") pod \"network-metrics-daemon-k4llc\" (UID: \"4411fcad-827e-4fe9-9165-8eaba1e75226\") " pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:47.448274 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448248    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-kubelet-config\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:47.448440 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448279    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-dbus\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:47.448440 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448325    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:47.448440 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448341    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-run\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.448440 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448365    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-var-lib-kubelet\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.448440 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448384    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-host\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.448440 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgdsz\" (UniqueName: \"kubernetes.io/projected/f41ef213-692e-4bc5-99f0-7dca93e13104-kube-api-access-zgdsz\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.448440 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-sysctl-d\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.448801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448446    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f41ef213-692e-4bc5-99f0-7dca93e13104-tmp\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.448801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448483    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-kubernetes\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.448801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448505    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-systemd\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.448801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448538    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-sysctl-conf\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.448801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448595    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-var-lib-kubelet\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.448801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-dbus\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:47.448801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448646    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-sysctl-d\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.448801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448670    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-sysctl-conf\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.448801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448742    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-host\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.448801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448787    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzbhq\" (UniqueName: \"kubernetes.io/projected/734454b5-e121-4543-ab79-ecfe0d985f96-kube-api-access-lzbhq\") pod \"node-resolver-89mdh\" (UID: \"734454b5-e121-4543-ab79-ecfe0d985f96\") " pod="openshift-dns/node-resolver-89mdh"
Jul 12 17:53:47.449265 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448818    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-tuned\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.449265 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-modprobe-d\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.449265 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448862    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-kubelet-config\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:47.449265 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448883    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-sysconfig\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.449265 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-sys\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.449265 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448925    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-run\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.449265 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448935    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-lib-modules\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.449265 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448964    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/734454b5-e121-4543-ab79-ecfe0d985f96-hosts-file\") pod \"node-resolver-89mdh\" (UID: \"734454b5-e121-4543-ab79-ecfe0d985f96\") " pod="openshift-dns/node-resolver-89mdh"
Jul 12 17:53:47.449265 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.448990    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/734454b5-e121-4543-ab79-ecfe0d985f96-tmp-dir\") pod \"node-resolver-89mdh\" (UID: \"734454b5-e121-4543-ab79-ecfe0d985f96\") " pod="openshift-dns/node-resolver-89mdh"
Jul 12 17:53:47.449265 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.449008    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 17:53:47.449265 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.449033    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-sys\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.449265 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.449072    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret podName:9d46ded0-0449-4d25-81bd-d3e7c5ae3a40 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:47.949053742 +0000 UTC m=+3.149173396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret") pod "global-pull-secret-syncer-97vpp" (UID: "9d46ded0-0449-4d25-81bd-d3e7c5ae3a40") : object "kube-system"/"original-pull-secret" not registered
Jul 12 17:53:47.449265 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.449095    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-sysconfig\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.449864 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.449281    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-modprobe-d\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.449864 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.449321    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-systemd\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.449864 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.449331    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/734454b5-e121-4543-ab79-ecfe0d985f96-hosts-file\") pod \"node-resolver-89mdh\" (UID: \"734454b5-e121-4543-ab79-ecfe0d985f96\") " pod="openshift-dns/node-resolver-89mdh"
Jul 12 17:53:47.449864 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.449373    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-kubernetes\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.449864 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.449426    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f41ef213-692e-4bc5-99f0-7dca93e13104-lib-modules\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.449864 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.449489    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/734454b5-e121-4543-ab79-ecfe0d985f96-tmp-dir\") pod \"node-resolver-89mdh\" (UID: \"734454b5-e121-4543-ab79-ecfe0d985f96\") " pod="openshift-dns/node-resolver-89mdh"
Jul 12 17:53:47.451333 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.451309    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f41ef213-692e-4bc5-99f0-7dca93e13104-etc-tuned\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.451944 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.451609    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f41ef213-692e-4bc5-99f0-7dca93e13104-tmp\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.459628 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.459584    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzbhq\" (UniqueName: \"kubernetes.io/projected/734454b5-e121-4543-ab79-ecfe0d985f96-kube-api-access-lzbhq\") pod \"node-resolver-89mdh\" (UID: \"734454b5-e121-4543-ab79-ecfe0d985f96\") " pod="openshift-dns/node-resolver-89mdh"
Jul 12 17:53:47.459843 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.459825    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgdsz\" (UniqueName: \"kubernetes.io/projected/f41ef213-692e-4bc5-99f0-7dca93e13104-kube-api-access-zgdsz\") pod \"tuned-vvzzz\" (UID: \"f41ef213-692e-4bc5-99f0-7dca93e13104\") " pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.472608 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.472573    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b2njf"
Jul 12 17:53:47.489146 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.489130    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:53:47.505080 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.505061    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jrpgp"
Jul 12 17:53:47.521472 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.521452    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fplg4"
Jul 12 17:53:47.531681 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.531224    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4"
Jul 12 17:53:47.544313 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.544283    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zgbc7"
Jul 12 17:53:47.556217 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.556200    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8wsqz"
Jul 12 17:53:47.576256 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.576238    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vvzzz"
Jul 12 17:53:47.580913 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.580897    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-89mdh"
Jul 12 17:53:47.631499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.631482    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 17:53:47.636093 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:47.636063    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b4433a31c1223ad0a8204daf58a4cd.slice/crio-7e4af190de70d50509424a8cc4bc44b26c5f84eee70c09fa226288d911cd1317 WatchSource:0}: Error finding container 7e4af190de70d50509424a8cc4bc44b26c5f84eee70c09fa226288d911cd1317: Status 404 returned error can't find the container with id 7e4af190de70d50509424a8cc4bc44b26c5f84eee70c09fa226288d911cd1317
Jul 12 17:53:47.636627 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:47.636610    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9235ba932a6066a0b11ef970f30044e4.slice/crio-0c2b7ce003e677583d656d55a55590df06bcf5805a34df7328f9faa36b198b38 WatchSource:0}: Error finding container 0c2b7ce003e677583d656d55a55590df06bcf5805a34df7328f9faa36b198b38: Status 404 returned error can't find the container with id 0c2b7ce003e677583d656d55a55590df06bcf5805a34df7328f9faa36b198b38
Jul 12 17:53:47.642411 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.642398    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 17:53:47.796773 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:47.796752    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd02b4365_fc9c_4ce5_ad3c_15b74ec8cd08.slice/crio-5d8bdf61cf269834bed8cb516630dee2fb3d5f293c4e546e45217ea5a261a35e WatchSource:0}: Error finding container 5d8bdf61cf269834bed8cb516630dee2fb3d5f293c4e546e45217ea5a261a35e: Status 404 returned error can't find the container with id 5d8bdf61cf269834bed8cb516630dee2fb3d5f293c4e546e45217ea5a261a35e
Jul 12 17:53:47.852458 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.852432    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs\") pod \"network-metrics-daemon-k4llc\" (UID: \"4411fcad-827e-4fe9-9165-8eaba1e75226\") " pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:47.852555 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.852469    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx29b\" (UniqueName: \"kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b\") pod \"network-check-target-ckkjw\" (UID: \"b4296c55-a6e8-4c34-93b6-3e72514d30bc\") " pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:53:47.852616 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.852578    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:53:47.852660 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.852638    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs podName:4411fcad-827e-4fe9-9165-8eaba1e75226 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:48.852624304 +0000 UTC m=+4.052743943 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs") pod "network-metrics-daemon-k4llc" (UID: "4411fcad-827e-4fe9-9165-8eaba1e75226") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:53:47.852713 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.852580    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 17:53:47.852713 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.852697    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 17:53:47.852713 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.852710    2578 projected.go:196] Error preparing data for projected volume kube-api-access-vx29b for pod openshift-network-diagnostics/network-check-target-ckkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:53:47.852816 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.852755    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b podName:b4296c55-a6e8-4c34-93b6-3e72514d30bc nodeName:}" failed. No retries permitted until 2026-07-12 17:53:48.852742525 +0000 UTC m=+4.052862163 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx29b" (UniqueName: "kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b") pod "network-check-target-ckkjw" (UID: "b4296c55-a6e8-4c34-93b6-3e72514d30bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:53:47.867969 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:47.867946    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca436ac_91ae_4edb_975c_98ee1847a28a.slice/crio-49702ebab7f7565d69f1a5e6770ce5e70fa25464bdb75607c079f8860c736397 WatchSource:0}: Error finding container 49702ebab7f7565d69f1a5e6770ce5e70fa25464bdb75607c079f8860c736397: Status 404 returned error can't find the container with id 49702ebab7f7565d69f1a5e6770ce5e70fa25464bdb75607c079f8860c736397
Jul 12 17:53:47.895575 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:47.895556    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61e71b08_89e9_4f55_a6db_a5f0dcd10f2a.slice/crio-be99eb701155e8ce8a4f7e0f22b67ccf10eacf1f4b5c7168846381adfef5b197 WatchSource:0}: Error finding container be99eb701155e8ce8a4f7e0f22b67ccf10eacf1f4b5c7168846381adfef5b197: Status 404 returned error can't find the container with id be99eb701155e8ce8a4f7e0f22b67ccf10eacf1f4b5c7168846381adfef5b197
Jul 12 17:53:47.896119 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:47.896095    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb092918b_d322_40b1_b850_33b856bb1ba4.slice/crio-1cab9f850935bdbda6fed3fe60f3c01d8438279d5bf1dd07011ae6f2c85f766f WatchSource:0}: Error finding container 1cab9f850935bdbda6fed3fe60f3c01d8438279d5bf1dd07011ae6f2c85f766f: Status 404 returned error can't find the container with id 1cab9f850935bdbda6fed3fe60f3c01d8438279d5bf1dd07011ae6f2c85f766f
Jul 12 17:53:47.945913 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:47.945877    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98dd2ed4_4bfb_4e7b_acae_e90695a81f22.slice/crio-42d9e349a9628b7974028f644e37427114d81429a460c84155329a4a209530ed WatchSource:0}: Error finding container 42d9e349a9628b7974028f644e37427114d81429a460c84155329a4a209530ed: Status 404 returned error can't find the container with id 42d9e349a9628b7974028f644e37427114d81429a460c84155329a4a209530ed
Jul 12 17:53:47.946548 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:47.946460    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod919b372b_2a09_4b02_a63b_79c46b645580.slice/crio-e1a3c187e1c85aa46ba1303f82ad19a25648e5673f58edf9d12a2075d1a1e1fc WatchSource:0}: Error finding container e1a3c187e1c85aa46ba1303f82ad19a25648e5673f58edf9d12a2075d1a1e1fc: Status 404 returned error can't find the container with id e1a3c187e1c85aa46ba1303f82ad19a25648e5673f58edf9d12a2075d1a1e1fc
Jul 12 17:53:47.949353 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:47.949329    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod734454b5_e121_4543_ab79_ecfe0d985f96.slice/crio-c58bd5669592659450e422b29df15e63081140faa7c423c78f939ce2d0e997d8 WatchSource:0}: Error finding container c58bd5669592659450e422b29df15e63081140faa7c423c78f939ce2d0e997d8: Status 404 returned error can't find the container with id c58bd5669592659450e422b29df15e63081140faa7c423c78f939ce2d0e997d8
Jul 12 17:53:47.953474 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:47.953453    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:47.953629 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.953612    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 17:53:47.953681 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:47.953664    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret podName:9d46ded0-0449-4d25-81bd-d3e7c5ae3a40 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:48.953649072 +0000 UTC m=+4.153768705 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret") pod "global-pull-secret-syncer-97vpp" (UID: "9d46ded0-0449-4d25-81bd-d3e7c5ae3a40") : object "kube-system"/"original-pull-secret" not registered
Jul 12 17:53:47.981141 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:47.981115    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a01ca45_b0df_41d8_9262_6a6f97d4a6ba.slice/crio-c1fd3fd1504cfe70ddcbe87dce70298542457543264c107cd240a122b7883ebe WatchSource:0}: Error finding container c1fd3fd1504cfe70ddcbe87dce70298542457543264c107cd240a122b7883ebe: Status 404 returned error can't find the container with id c1fd3fd1504cfe70ddcbe87dce70298542457543264c107cd240a122b7883ebe
Jul 12 17:53:48.039970 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:53:48.039949    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf41ef213_692e_4bc5_99f0_7dca93e13104.slice/crio-26c1997689ea9cb5f885342064541ce6735dedbd0e6a1d5762d818ee3c4935b4 WatchSource:0}: Error finding container 26c1997689ea9cb5f885342064541ce6735dedbd0e6a1d5762d818ee3c4935b4: Status 404 returned error can't find the container with id 26c1997689ea9cb5f885342064541ce6735dedbd0e6a1d5762d818ee3c4935b4
Jul 12 17:53:48.199010 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.198891    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 17:48:47 +0000 UTC" deadline="2028-04-13 21:30:20.614045754 +0000 UTC"
Jul 12 17:53:48.199010 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.198950    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15387h36m32.415099751s"
Jul 12 17:53:48.320107 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.320074    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:53:48.320280 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:48.320206    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:53:48.472203 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.472031    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vvzzz" event={"ID":"f41ef213-692e-4bc5-99f0-7dca93e13104","Type":"ContainerStarted","Data":"26c1997689ea9cb5f885342064541ce6735dedbd0e6a1d5762d818ee3c4935b4"}
Jul 12 17:53:48.480184 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.480094    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-89mdh" event={"ID":"734454b5-e121-4543-ab79-ecfe0d985f96","Type":"ContainerStarted","Data":"c58bd5669592659450e422b29df15e63081140faa7c423c78f939ce2d0e997d8"}
Jul 12 17:53:48.494332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.494278    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8wsqz" event={"ID":"98dd2ed4-4bfb-4e7b-acae-e90695a81f22","Type":"ContainerStarted","Data":"42d9e349a9628b7974028f644e37427114d81429a460c84155329a4a209530ed"}
Jul 12 17:53:48.506875 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.506845    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b2njf" event={"ID":"b092918b-d322-40b1-b850-33b856bb1ba4","Type":"ContainerStarted","Data":"1cab9f850935bdbda6fed3fe60f3c01d8438279d5bf1dd07011ae6f2c85f766f"}
Jul 12 17:53:48.529655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.529626    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fplg4" event={"ID":"9ca436ac-91ae-4edb-975c-98ee1847a28a","Type":"ContainerStarted","Data":"49702ebab7f7565d69f1a5e6770ce5e70fa25464bdb75607c079f8860c736397"}
Jul 12 17:53:48.544431 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.544402    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl" event={"ID":"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08","Type":"ContainerStarted","Data":"5d8bdf61cf269834bed8cb516630dee2fb3d5f293c4e546e45217ea5a261a35e"}
Jul 12 17:53:48.556110 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.556081    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-122.ec2.internal" event={"ID":"9235ba932a6066a0b11ef970f30044e4","Type":"ContainerStarted","Data":"0c2b7ce003e677583d656d55a55590df06bcf5805a34df7328f9faa36b198b38"}
Jul 12 17:53:48.561058 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.561031    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zgbc7" event={"ID":"8a01ca45-b0df-41d8-9262-6a6f97d4a6ba","Type":"ContainerStarted","Data":"c1fd3fd1504cfe70ddcbe87dce70298542457543264c107cd240a122b7883ebe"}
Jul 12 17:53:48.565968 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.565935    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4" event={"ID":"919b372b-2a09-4b02-a63b-79c46b645580","Type":"ContainerStarted","Data":"e1a3c187e1c85aa46ba1303f82ad19a25648e5673f58edf9d12a2075d1a1e1fc"}
Jul 12 17:53:48.579819 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.579762    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jrpgp" event={"ID":"61e71b08-89e9-4f55-a6db-a5f0dcd10f2a","Type":"ContainerStarted","Data":"be99eb701155e8ce8a4f7e0f22b67ccf10eacf1f4b5c7168846381adfef5b197"}
Jul 12 17:53:48.585314 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.585274    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal" event={"ID":"b7b4433a31c1223ad0a8204daf58a4cd","Type":"ContainerStarted","Data":"7e4af190de70d50509424a8cc4bc44b26c5f84eee70c09fa226288d911cd1317"}
Jul 12 17:53:48.860080 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.859994    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs\") pod \"network-metrics-daemon-k4llc\" (UID: \"4411fcad-827e-4fe9-9165-8eaba1e75226\") " pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:48.860080 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.860050    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx29b\" (UniqueName: \"kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b\") pod \"network-check-target-ckkjw\" (UID: \"b4296c55-a6e8-4c34-93b6-3e72514d30bc\") " pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:53:48.860344 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:48.860209    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 17:53:48.860344 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:48.860229    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 17:53:48.860344 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:48.860241    2578 projected.go:196] Error preparing data for projected volume kube-api-access-vx29b for pod openshift-network-diagnostics/network-check-target-ckkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:53:48.860344 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:48.860305    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b podName:b4296c55-a6e8-4c34-93b6-3e72514d30bc nodeName:}" failed. No retries permitted until 2026-07-12 17:53:50.860285021 +0000 UTC m=+6.060404657 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx29b" (UniqueName: "kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b") pod "network-check-target-ckkjw" (UID: "b4296c55-a6e8-4c34-93b6-3e72514d30bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:53:48.860626 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:48.860384    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:53:48.860626 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:48.860421    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs podName:4411fcad-827e-4fe9-9165-8eaba1e75226 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:50.860409001 +0000 UTC m=+6.060528635 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs") pod "network-metrics-daemon-k4llc" (UID: "4411fcad-827e-4fe9-9165-8eaba1e75226") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:53:48.961506 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:48.961467    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:48.961699 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:48.961682    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 17:53:48.962343 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:48.961786    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret podName:9d46ded0-0449-4d25-81bd-d3e7c5ae3a40 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:50.961765004 +0000 UTC m=+6.161884642 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret") pod "global-pull-secret-syncer-97vpp" (UID: "9d46ded0-0449-4d25-81bd-d3e7c5ae3a40") : object "kube-system"/"original-pull-secret" not registered
Jul 12 17:53:49.062346 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:49.062316    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 17:53:49.181699 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:49.181660    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 12 17:53:49.199839 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:49.199769    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 17:48:47 +0000 UTC" deadline="2028-01-28 17:54:40.773445528 +0000 UTC"
Jul 12 17:53:49.199839 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:49.199840    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13560h0m51.573610446s"
Jul 12 17:53:49.321407 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:49.320739    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:49.321407 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:49.320885    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:53:49.321407 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:49.320739    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:49.321407 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:49.321351    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:53:50.320326 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:50.319745    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:53:50.320326 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:50.319876    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:53:50.880147 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:50.879311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx29b\" (UniqueName: \"kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b\") pod \"network-check-target-ckkjw\" (UID: \"b4296c55-a6e8-4c34-93b6-3e72514d30bc\") " pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:53:50.880147 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:50.879409    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs\") pod \"network-metrics-daemon-k4llc\" (UID: \"4411fcad-827e-4fe9-9165-8eaba1e75226\") " pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:50.880147 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:50.879534    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:53:50.880147 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:50.879626    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs podName:4411fcad-827e-4fe9-9165-8eaba1e75226 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:54.879605522 +0000 UTC m=+10.079725171 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs") pod "network-metrics-daemon-k4llc" (UID: "4411fcad-827e-4fe9-9165-8eaba1e75226") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:53:50.880147 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:50.879728    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 17:53:50.880147 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:50.879743    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 17:53:50.880147 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:50.879756    2578 projected.go:196] Error preparing data for projected volume kube-api-access-vx29b for pod openshift-network-diagnostics/network-check-target-ckkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:53:50.880147 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:50.879791    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b podName:b4296c55-a6e8-4c34-93b6-3e72514d30bc nodeName:}" failed. No retries permitted until 2026-07-12 17:53:54.879779886 +0000 UTC m=+10.079899523 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx29b" (UniqueName: "kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b") pod "network-check-target-ckkjw" (UID: "b4296c55-a6e8-4c34-93b6-3e72514d30bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:53:50.980482 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:50.980442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:50.980685 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:50.980662    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 17:53:50.980802 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:50.980723    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret podName:9d46ded0-0449-4d25-81bd-d3e7c5ae3a40 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:54.980704355 +0000 UTC m=+10.180823996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret") pod "global-pull-secret-syncer-97vpp" (UID: "9d46ded0-0449-4d25-81bd-d3e7c5ae3a40") : object "kube-system"/"original-pull-secret" not registered
Jul 12 17:53:51.319922 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:51.319888    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:51.320107 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:51.320034    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:53:51.320521 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:51.320498    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:51.320954 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:51.320620    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:53:52.320087 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:52.320047    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:53:52.320289 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:52.320184    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:53:53.320076 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:53.320044    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:53.320509 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:53.320206    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:53:53.320509 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:53.320259    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:53.320509 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:53.320392    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:53:54.319614 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:54.319586    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:53:54.319798 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:54.319685    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:53:54.914184 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:54.914146    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx29b\" (UniqueName: \"kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b\") pod \"network-check-target-ckkjw\" (UID: \"b4296c55-a6e8-4c34-93b6-3e72514d30bc\") " pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:53:54.914637 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:54.914246    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs\") pod \"network-metrics-daemon-k4llc\" (UID: \"4411fcad-827e-4fe9-9165-8eaba1e75226\") " pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:54.914637 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:54.914310    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 17:53:54.914637 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:54.914348    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 17:53:54.914637 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:54.914362    2578 projected.go:196] Error preparing data for projected volume kube-api-access-vx29b for pod openshift-network-diagnostics/network-check-target-ckkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:53:54.914637 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:54.914391    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:53:54.914637 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:54.914434    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b podName:b4296c55-a6e8-4c34-93b6-3e72514d30bc nodeName:}" failed. No retries permitted until 2026-07-12 17:54:02.914414712 +0000 UTC m=+18.114534366 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx29b" (UniqueName: "kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b") pod "network-check-target-ckkjw" (UID: "b4296c55-a6e8-4c34-93b6-3e72514d30bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:53:54.914637 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:54.914457    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs podName:4411fcad-827e-4fe9-9165-8eaba1e75226 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:02.914446855 +0000 UTC m=+18.114566487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs") pod "network-metrics-daemon-k4llc" (UID: "4411fcad-827e-4fe9-9165-8eaba1e75226") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:53:55.015027 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:55.014983    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:55.015194 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:55.015143    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 17:53:55.015262 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:55.015224    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret podName:9d46ded0-0449-4d25-81bd-d3e7c5ae3a40 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:03.015201416 +0000 UTC m=+18.215321049 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret") pod "global-pull-secret-syncer-97vpp" (UID: "9d46ded0-0449-4d25-81bd-d3e7c5ae3a40") : object "kube-system"/"original-pull-secret" not registered
Jul 12 17:53:55.320981 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:55.320948    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:55.321117 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:55.321068    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:53:55.321529 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:55.321502    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:55.321644 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:55.321622    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:53:56.319703 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:56.319672    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:53:56.320149 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:56.319793    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:53:57.319410 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:57.319375    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:57.319570 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:57.319427    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:57.319570 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:57.319550    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:53:57.319695 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:57.319677    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:53:58.319548 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:58.319491    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:53:58.319922 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:58.319667    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:53:59.319467 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:59.319436    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:53:59.319641 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:53:59.319436    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:53:59.319641 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:59.319580    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:53:59.320020 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:53:59.319693    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:00.319766 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:00.319733    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:00.320149 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:00.319839    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:01.319775 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:01.319741    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:01.320201 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:01.319882    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:01.320201 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:01.319945    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:01.320201 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:01.320062    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:02.320326 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:02.320124    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:02.320735 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:02.320420    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:02.976351 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:02.976321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs\") pod \"network-metrics-daemon-k4llc\" (UID: \"4411fcad-827e-4fe9-9165-8eaba1e75226\") " pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:02.976561 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:02.976360    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx29b\" (UniqueName: \"kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b\") pod \"network-check-target-ckkjw\" (UID: \"b4296c55-a6e8-4c34-93b6-3e72514d30bc\") " pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:02.976561 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:02.976466    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:54:02.976561 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:02.976549    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs podName:4411fcad-827e-4fe9-9165-8eaba1e75226 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:18.976513387 +0000 UTC m=+34.176633019 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs") pod "network-metrics-daemon-k4llc" (UID: "4411fcad-827e-4fe9-9165-8eaba1e75226") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:54:02.976561 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:02.976470    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 17:54:02.976772 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:02.976573    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 17:54:02.976772 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:02.976585    2578 projected.go:196] Error preparing data for projected volume kube-api-access-vx29b for pod openshift-network-diagnostics/network-check-target-ckkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:54:02.976772 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:02.976630    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b podName:b4296c55-a6e8-4c34-93b6-3e72514d30bc nodeName:}" failed. No retries permitted until 2026-07-12 17:54:18.976617441 +0000 UTC m=+34.176737074 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx29b" (UniqueName: "kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b") pod "network-check-target-ckkjw" (UID: "b4296c55-a6e8-4c34-93b6-3e72514d30bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:54:03.076953 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:03.076922    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:03.077103 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:03.077071    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 17:54:03.077167 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:03.077130    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret podName:9d46ded0-0449-4d25-81bd-d3e7c5ae3a40 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:19.077116051 +0000 UTC m=+34.277235687 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret") pod "global-pull-secret-syncer-97vpp" (UID: "9d46ded0-0449-4d25-81bd-d3e7c5ae3a40") : object "kube-system"/"original-pull-secret" not registered
Jul 12 17:54:03.319496 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:03.319422    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:03.319654 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:03.319496    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:03.319654 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:03.319644    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:03.319789 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:03.319766    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:04.320022 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:04.319985    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:04.320440 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:04.320106    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:05.320696 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:05.320656    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:05.321100 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:05.320759    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:05.321100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:05.320847    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:05.321100 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:05.320935    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:06.319294 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:06.319262    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:06.319446 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:06.319369    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:07.320033 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:07.320006    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:07.320465 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:07.320112    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:07.320465 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:07.320181    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:07.320465 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:07.320265    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:08.319870 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.319725    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:08.319963 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:08.319930    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:08.638037 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.637728    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jrpgp" event={"ID":"61e71b08-89e9-4f55-a6db-a5f0dcd10f2a","Type":"ContainerStarted","Data":"b53fb5d79f0d89ceec209d21d4c6f915c97e5bc2e06774d62d03f1aaa1718f67"}
Jul 12 17:54:08.639417 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.639393    2578 generic.go:356] "Generic (PLEG): container finished" podID="b7b4433a31c1223ad0a8204daf58a4cd" containerID="8b24f7803072addc458fd84dca99ec2853098d7197507571f41376888aa5d144" exitCode=0
Jul 12 17:54:08.639553 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.639461    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal" event={"ID":"b7b4433a31c1223ad0a8204daf58a4cd","Type":"ContainerDied","Data":"8b24f7803072addc458fd84dca99ec2853098d7197507571f41376888aa5d144"}
Jul 12 17:54:08.641116 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.641089    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vvzzz" event={"ID":"f41ef213-692e-4bc5-99f0-7dca93e13104","Type":"ContainerStarted","Data":"602e064d08735b0ee68f20724171b19797c1d745161e6235dd9d6a181ba30c25"}
Jul 12 17:54:08.643022 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.642998    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-89mdh" event={"ID":"734454b5-e121-4543-ab79-ecfe0d985f96","Type":"ContainerStarted","Data":"e3bfb400d2bfc19e530a40b1177dfb85fb3030e194add2963ad356709af30275"}
Jul 12 17:54:08.648209 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.648132    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8wsqz" event={"ID":"98dd2ed4-4bfb-4e7b-acae-e90695a81f22","Type":"ContainerStarted","Data":"6bca6f50316224534dc403b77b738c12f12ea195e54163b03a3fbad837005303"}
Jul 12 17:54:08.650397 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.650372    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b2njf" event={"ID":"b092918b-d322-40b1-b850-33b856bb1ba4","Type":"ContainerStarted","Data":"7309a492fd651ea580e14100ea6c882440ffc1474cf5101350a4c4cb688e6369"}
Jul 12 17:54:08.652254 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.652234    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ca436ac-91ae-4edb-975c-98ee1847a28a" containerID="a1752f55f3a64d6734723219b76de47a772a49498983f0d941db8444248a1c64" exitCode=0
Jul 12 17:54:08.652318 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.652306    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fplg4" event={"ID":"9ca436ac-91ae-4edb-975c-98ee1847a28a","Type":"ContainerDied","Data":"a1752f55f3a64d6734723219b76de47a772a49498983f0d941db8444248a1c64"}
Jul 12 17:54:08.656023 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.656006    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 17:54:08.656392 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.656369    2578 generic.go:356] "Generic (PLEG): container finished" podID="d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08" containerID="15cab9eba97d9ca346c9cf89b1f45f6da450ffda4e64ec41020fc2c8f3d962ed" exitCode=1
Jul 12 17:54:08.656486 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.656434    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl" event={"ID":"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08","Type":"ContainerStarted","Data":"ed613754607d555cfc296efaa80c4160e9e289dd55550cc56af2841c575250f6"}
Jul 12 17:54:08.656486 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.656482    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl" event={"ID":"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08","Type":"ContainerStarted","Data":"729a72c31178abfc72882a19959b143d6383f18c7e6c504445649a0d2db22df0"}
Jul 12 17:54:08.656602 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.656571    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl" event={"ID":"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08","Type":"ContainerStarted","Data":"55c12c61599c7416b11fa48a9eff7794cf040bb1bf6cf0d447d8fbdb3dd7f207"}
Jul 12 17:54:08.656602 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.656600    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl" event={"ID":"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08","Type":"ContainerStarted","Data":"9c31db9ae80bf2c9fa30b78aa1bd557d4d6bf07cba0f5d297c3c5dfa99010875"}
Jul 12 17:54:08.656708 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.656627    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl" event={"ID":"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08","Type":"ContainerDied","Data":"15cab9eba97d9ca346c9cf89b1f45f6da450ffda4e64ec41020fc2c8f3d962ed"}
Jul 12 17:54:08.656708 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.656668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl" event={"ID":"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08","Type":"ContainerStarted","Data":"2d78b15cd723ef017ca3fe37a30eb5868c428fa254c380bb8e289306d3be10f9"}
Jul 12 17:54:08.657971 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.657949    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-122.ec2.internal" event={"ID":"9235ba932a6066a0b11ef970f30044e4","Type":"ContainerStarted","Data":"62c994c990e007f289db302ebbf6475bf860016575bfefa681a42c9dffbd7579"}
Jul 12 17:54:08.658560 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.658501    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jrpgp" podStartSLOduration=2.860118258 podStartE2EDuration="22.658485765s" totalImagesPullingTime="19.798367507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:08.657893334 +0000 UTC m=+23.858012987" watchObservedRunningTime="2026-07-12 17:54:08.658485765 +0000 UTC m=+23.858605419"
Jul 12 17:54:08.659448 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.659429    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4" event={"ID":"919b372b-2a09-4b02-a63b-79c46b645580","Type":"ContainerStarted","Data":"9e21c1d9576010513febb66bcc533a8fdf8bdcc6dfbc88578480cd8c8e1fca4e"}
Jul 12 17:54:08.679727 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.679683    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vvzzz" podStartSLOduration=3.020167468 podStartE2EDuration="22.679645144s" totalImagesPullingTime="19.659477676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:08.679210994 +0000 UTC m=+23.879330648" watchObservedRunningTime="2026-07-12 17:54:08.679645144 +0000 UTC m=+23.879764800"
Jul 12 17:54:08.724273 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.724238    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-89mdh" podStartSLOduration=2.996479708 podStartE2EDuration="22.724225795s" totalImagesPullingTime="19.727746087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:08.698227412 +0000 UTC m=+23.898347067" watchObservedRunningTime="2026-07-12 17:54:08.724225795 +0000 UTC m=+23.924345473"
Jul 12 17:54:08.767783 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.767716    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8wsqz" podStartSLOduration=2.859892843 podStartE2EDuration="22.767670289s" totalImagesPullingTime="19.907777446s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:08.751771242 +0000 UTC m=+23.951890895" watchObservedRunningTime="2026-07-12 17:54:08.767670289 +0000 UTC m=+23.967789945"
Jul 12 17:54:08.782291 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:08.782249    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-122.ec2.internal" podStartSLOduration=21.7822326 podStartE2EDuration="21.7822326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:08.782062 +0000 UTC m=+23.982181654" watchObservedRunningTime="2026-07-12 17:54:08.7822326 +0000 UTC m=+23.982352254"
Jul 12 17:54:09.300013 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:09.299986    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 12 17:54:09.320067 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:09.320043    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:09.320237 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:09.320185    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:09.320284 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:09.320241    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:09.320377 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:09.320353    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:09.663853 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:09.663778    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zgbc7" event={"ID":"8a01ca45-b0df-41d8-9262-6a6f97d4a6ba","Type":"ContainerStarted","Data":"b1c36c7ba5be58c93d152b8694a3844876ce191cd0962d4d49e843fd6b8dfd1a"}
Jul 12 17:54:09.665928 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:09.665904    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4" event={"ID":"919b372b-2a09-4b02-a63b-79c46b645580","Type":"ContainerStarted","Data":"0349738bf26c497bc7d2b19e2f91e7a233131e9da63b4ad6a6c0351257f99a91"}
Jul 12 17:54:09.667746 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:09.667709    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal" event={"ID":"b7b4433a31c1223ad0a8204daf58a4cd","Type":"ContainerStarted","Data":"9fe9a5df352de8ba6795f0193de448664583494c59a91325b43ce8f55c5aa381"}
Jul 12 17:54:09.678716 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:09.678672    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-b2njf" podStartSLOduration=3.901818517 podStartE2EDuration="23.67864776s" totalImagesPullingTime="19.776829243s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:08.799226846 +0000 UTC m=+23.999346499" watchObservedRunningTime="2026-07-12 17:54:09.67864776 +0000 UTC m=+24.878767413"
Jul 12 17:54:09.678863 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:09.678838    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zgbc7" podStartSLOduration=3.981625396 podStartE2EDuration="23.67883011s" totalImagesPullingTime="19.697204714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:09.678571149 +0000 UTC m=+24.878690803" watchObservedRunningTime="2026-07-12 17:54:09.67883011 +0000 UTC m=+24.878949756"
Jul 12 17:54:09.696589 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:09.695559    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-122.ec2.internal" podStartSLOduration=22.695545219 podStartE2EDuration="22.695545219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:09.694812389 +0000 UTC m=+24.894932041" watchObservedRunningTime="2026-07-12 17:54:09.695545219 +0000 UTC m=+24.895664873"
Jul 12 17:54:10.240313 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:10.240228    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T17:54:09.300010163Z","UUID":"2c2843b5-f416-4fa2-a3df-8cde4c4f6333","Handler":null,"Name":"","Endpoint":""}
Jul 12 17:54:10.242690 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:10.242669    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 12 17:54:10.242820 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:10.242707    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 12 17:54:10.319345 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:10.319312    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:10.319471 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:10.319432    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:10.674679 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:10.674657    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 17:54:10.675140 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:10.675110    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl" event={"ID":"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08","Type":"ContainerStarted","Data":"7ecd30b7118a0313a363d9b326d2777c1019abde64ed5a1d12ac1e2dc89047ce"}
Jul 12 17:54:10.677631 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:10.677594    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4" event={"ID":"919b372b-2a09-4b02-a63b-79c46b645580","Type":"ContainerStarted","Data":"9911c2877259c92943386a46ea5783c708dd5360636e95e22005bb3ba727082b"}
Jul 12 17:54:10.696296 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:10.696247    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjsl4" podStartSLOduration=2.815539219 podStartE2EDuration="24.696233338s" totalImagesPullingTime="21.880694119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:10.695857415 +0000 UTC m=+25.895977068" watchObservedRunningTime="2026-07-12 17:54:10.696233338 +0000 UTC m=+25.896352991"
Jul 12 17:54:11.319457 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:11.319427    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:11.319743 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:11.319571    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:11.319743 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:11.319628    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:11.319743 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:11.319730    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:12.320296 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:12.320074    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:12.320742 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:12.320376    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:12.455994 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:12.455962    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jrpgp"
Jul 12 17:54:12.456821 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:12.456801    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jrpgp"
Jul 12 17:54:12.681406 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:12.681378    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jrpgp"
Jul 12 17:54:12.682006 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:12.681981    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jrpgp"
Jul 12 17:54:13.319967 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:13.319808    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:13.320059 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:13.319856    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:13.320093 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:13.320055    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:13.320185 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:13.320162    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:13.684271 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:13.684245    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ca436ac-91ae-4edb-975c-98ee1847a28a" containerID="4bd3f03d990cd9013068ca1fc395dc2cc0965a1a2ff687a3c5c3bb3c3dca1474" exitCode=0
Jul 12 17:54:13.684944 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:13.684326    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fplg4" event={"ID":"9ca436ac-91ae-4edb-975c-98ee1847a28a","Type":"ContainerDied","Data":"4bd3f03d990cd9013068ca1fc395dc2cc0965a1a2ff687a3c5c3bb3c3dca1474"}
Jul 12 17:54:13.687578 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:13.687560    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 17:54:13.687988 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:13.687970    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl" event={"ID":"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08","Type":"ContainerStarted","Data":"61d172751463ec35ec8736741a548df764032893d2d71cd76170543c3de26c70"}
Jul 12 17:54:13.688330 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:13.688317    2578 scope.go:117] "RemoveContainer" containerID="15cab9eba97d9ca346c9cf89b1f45f6da450ffda4e64ec41020fc2c8f3d962ed"
Jul 12 17:54:14.319866 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:14.319841    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:14.320022 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:14.319937    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:14.694048 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:14.694013    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 17:54:14.694463 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:14.694432    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl" event={"ID":"d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08","Type":"ContainerStarted","Data":"2cd599b4989e805eb63cb0eb8c57d4977db4dcce73fadda75c177b3946004f2e"}
Jul 12 17:54:14.694936 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:14.694911    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:54:14.695070 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:14.694984    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:54:14.695070 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:14.695031    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:54:14.711373 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:14.711350    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:54:14.711480 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:14.711448    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:54:14.725386 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:14.725343    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl" podStartSLOduration=8.801926502 podStartE2EDuration="28.725330771s" totalImagesPullingTime="19.923404269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:14.72489245 +0000 UTC m=+29.925012114" watchObservedRunningTime="2026-07-12 17:54:14.725330771 +0000 UTC m=+29.925450425"
Jul 12 17:54:15.320772 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:15.320742    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:15.320926 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:15.320834    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:15.320926 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:15.320913    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:15.321046 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:15.320993    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:15.698672 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:15.698633    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ca436ac-91ae-4edb-975c-98ee1847a28a" containerID="c02ce432b8afcd0bf750077f3202b9c0cd100a4f68611f79089272eb4895eaaa" exitCode=0
Jul 12 17:54:15.699066 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:15.698713    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fplg4" event={"ID":"9ca436ac-91ae-4edb-975c-98ee1847a28a","Type":"ContainerDied","Data":"c02ce432b8afcd0bf750077f3202b9c0cd100a4f68611f79089272eb4895eaaa"}
Jul 12 17:54:16.319883 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:16.319849    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:16.320024 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:16.319951    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:17.319611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:17.319580    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:17.320021 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:17.319703    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:17.320021 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:17.319757    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:17.320021 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:17.319860    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:17.704618 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:17.704589    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ca436ac-91ae-4edb-975c-98ee1847a28a" containerID="d815e63c4a2dad3b2214efd74e55851db913c6e21475a5a8f3e32769a8f84404" exitCode=0
Jul 12 17:54:17.704745 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:17.704647    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fplg4" event={"ID":"9ca436ac-91ae-4edb-975c-98ee1847a28a","Type":"ContainerDied","Data":"d815e63c4a2dad3b2214efd74e55851db913c6e21475a5a8f3e32769a8f84404"}
Jul 12 17:54:18.319499 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:18.319469    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:18.319660 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:18.319599    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:18.987313 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:18.987275    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs\") pod \"network-metrics-daemon-k4llc\" (UID: \"4411fcad-827e-4fe9-9165-8eaba1e75226\") " pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:18.987503 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:18.987328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx29b\" (UniqueName: \"kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b\") pod \"network-check-target-ckkjw\" (UID: \"b4296c55-a6e8-4c34-93b6-3e72514d30bc\") " pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:18.987503 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:18.987414    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:54:18.987503 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:18.987445    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 12 17:54:18.987503 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:18.987461    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 12 17:54:18.987503 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:18.987473    2578 projected.go:196] Error preparing data for projected volume kube-api-access-vx29b for pod openshift-network-diagnostics/network-check-target-ckkjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:54:18.987503 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:18.987488    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs podName:4411fcad-827e-4fe9-9165-8eaba1e75226 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:50.987466454 +0000 UTC m=+66.187586110 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs") pod "network-metrics-daemon-k4llc" (UID: "4411fcad-827e-4fe9-9165-8eaba1e75226") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 12 17:54:18.987852 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:18.987528    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b podName:b4296c55-a6e8-4c34-93b6-3e72514d30bc nodeName:}" failed. No retries permitted until 2026-07-12 17:54:50.987500532 +0000 UTC m=+66.187620183 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vx29b" (UniqueName: "kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b") pod "network-check-target-ckkjw" (UID: "b4296c55-a6e8-4c34-93b6-3e72514d30bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 12 17:54:19.088218 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:19.088192    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:19.088362 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:19.088330    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 12 17:54:19.088422 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:19.088383    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret podName:9d46ded0-0449-4d25-81bd-d3e7c5ae3a40 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:51.088369694 +0000 UTC m=+66.288489332 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret") pod "global-pull-secret-syncer-97vpp" (UID: "9d46ded0-0449-4d25-81bd-d3e7c5ae3a40") : object "kube-system"/"original-pull-secret" not registered
Jul 12 17:54:19.319637 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:19.319570    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:19.319637 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:19.319592    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:19.320092 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:19.319697    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:19.320092 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:19.319839    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:20.320092 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:20.320062    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:20.320438 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:20.320152    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:21.319570 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:21.319536    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:21.319570 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:21.319551    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:21.319851 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:21.319665    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:21.319851 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:21.319804    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:22.319595 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:22.319570    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:22.319882 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:22.319668    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:22.718016 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:22.717988    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ca436ac-91ae-4edb-975c-98ee1847a28a" containerID="a076c29d600fb4441f4e90085ec3ac0ee6cd9aa194e8549b02b212568fee6748" exitCode=0
Jul 12 17:54:22.718138 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:22.718028    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fplg4" event={"ID":"9ca436ac-91ae-4edb-975c-98ee1847a28a","Type":"ContainerDied","Data":"a076c29d600fb4441f4e90085ec3ac0ee6cd9aa194e8549b02b212568fee6748"}
Jul 12 17:54:23.319904 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:23.319876    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:23.320233 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:23.319975    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:23.320233 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:23.320051    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:23.320233 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:23.320128    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:23.722852 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:23.722824    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ca436ac-91ae-4edb-975c-98ee1847a28a" containerID="84fcd32430a36d407a9fe087591540fabbc1c55730db4d830ab127081aa6fa19" exitCode=0
Jul 12 17:54:23.722987 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:23.722870    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fplg4" event={"ID":"9ca436ac-91ae-4edb-975c-98ee1847a28a","Type":"ContainerDied","Data":"84fcd32430a36d407a9fe087591540fabbc1c55730db4d830ab127081aa6fa19"}
Jul 12 17:54:24.319550 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:24.319510    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:24.319682 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:24.319607    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:24.728207 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:24.728170    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fplg4" event={"ID":"9ca436ac-91ae-4edb-975c-98ee1847a28a","Type":"ContainerStarted","Data":"7170da16a4a874507c5435f6cd348c8d93f1ff424782e58403682616fde1e86f"}
Jul 12 17:54:24.751757 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:24.751720    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fplg4" podStartSLOduration=7.627929376 podStartE2EDuration="38.751707653s" totalImagesPullingTime="31.123778277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:24.751155054 +0000 UTC m=+39.951274708" watchObservedRunningTime="2026-07-12 17:54:24.751707653 +0000 UTC m=+39.951827306"
Jul 12 17:54:25.321024 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:25.320798    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:25.321186 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:25.320845    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:25.321186 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:25.321119    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:25.321186 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:25.321161    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:26.319661 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:26.319633    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:26.319999 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:26.319713    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:27.319231 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:27.319197    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:27.319398 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:27.319315    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:27.319398 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:27.319364    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:27.319509 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:27.319455    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:28.319643 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:28.319610    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:28.319971 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:28.319715    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:29.319638 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:29.319608    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:29.319800 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:29.319708    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:29.319800 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:29.319790    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:29.320146 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:29.319923    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:30.319444 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:30.319413    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:30.319672 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:30.319508    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:31.319506 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:31.319482    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:31.320028 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:31.319609    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:31.320028 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:31.319662    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:31.320028 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:31.319771    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:31.415380 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:31.415353    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-97vpp"]
Jul 12 17:54:31.418040 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:31.418017    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k4llc"]
Jul 12 17:54:31.429705 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:31.429682    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ckkjw"]
Jul 12 17:54:31.429821 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:31.429775    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:31.429902 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:31.429878    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:31.742147 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:31.742116    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:31.742280 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:31.742115    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:31.742280 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:31.742218    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:31.742351 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:31.742304    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:33.319567 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:33.319540    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:33.319939 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:33.319542    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:33.319939 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:33.319639    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:33.319939 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:33.319547    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:33.319939 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:33.319710    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:33.319939 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:33.319817    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:35.320100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.320076    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:35.320444 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:35.320162    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k4llc" podUID="4411fcad-827e-4fe9-9165-8eaba1e75226"
Jul 12 17:54:35.320444 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.320270    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:35.320444 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:35.320395    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-97vpp" podUID="9d46ded0-0449-4d25-81bd-d3e7c5ae3a40"
Jul 12 17:54:35.320444 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.320432    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:35.320670 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:35.320493    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ckkjw" podUID="b4296c55-a6e8-4c34-93b6-3e72514d30bc"
Jul 12 17:54:35.623022 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.622949    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-122.ec2.internal" event="NodeReady"
Jul 12 17:54:35.623138 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.623059    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 12 17:54:35.668612 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.668584    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pmx65"]
Jul 12 17:54:35.684534 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.684279    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8xmkt"]
Jul 12 17:54:35.684766 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.684429    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:35.687093 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.687069    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 12 17:54:35.687322 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.687309    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtknx\""
Jul 12 17:54:35.687322 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.687339    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 12 17:54:35.696130 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.696113    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pmx65"]
Jul 12 17:54:35.696202 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.696144    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8xmkt"]
Jul 12 17:54:35.696241 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.696225    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.698768 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.698748    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 12 17:54:35.698768 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.698757    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q8r2n\""
Jul 12 17:54:35.698910 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.698764    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 12 17:54:35.698910 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.698850    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 12 17:54:35.699005 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.698974    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 12 17:54:35.769410 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.769384    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hzhk4"]
Jul 12 17:54:35.776803 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.776786    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hzhk4"
Jul 12 17:54:35.779216 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.779195    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 12 17:54:35.779327 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.779268    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-59fr9\""
Jul 12 17:54:35.779327 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.779278    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 12 17:54:35.779424 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.779367    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 12 17:54:35.782838 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.782820    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hzhk4"]
Jul 12 17:54:35.805870 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.805845    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fmkq\" (UniqueName: \"kubernetes.io/projected/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-api-access-5fmkq\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.805948 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.805872    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19eadb03-b4c4-4f60-a60e-db68e797d2ad-tmp-dir\") pod \"dns-default-pmx65\" (UID: \"19eadb03-b4c4-4f60-a60e-db68e797d2ad\") " pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:35.805948 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.805891    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f14ee62-aa17-47d0-8e04-eb49058e0e91-crio-socket\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.805948 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.805911    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19eadb03-b4c4-4f60-a60e-db68e797d2ad-config-volume\") pod \"dns-default-pmx65\" (UID: \"19eadb03-b4c4-4f60-a60e-db68e797d2ad\") " pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:35.805948 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.805930    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19eadb03-b4c4-4f60-a60e-db68e797d2ad-metrics-tls\") pod \"dns-default-pmx65\" (UID: \"19eadb03-b4c4-4f60-a60e-db68e797d2ad\") " pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:35.806103 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.805983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f14ee62-aa17-47d0-8e04-eb49058e0e91-data-volume\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.806103 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.806019    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r29lc\" (UniqueName: \"kubernetes.io/projected/19eadb03-b4c4-4f60-a60e-db68e797d2ad-kube-api-access-r29lc\") pod \"dns-default-pmx65\" (UID: \"19eadb03-b4c4-4f60-a60e-db68e797d2ad\") " pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:35.806103 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.806089    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.806276 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.806111    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.907224 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907160    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eea7d684-a24b-4464-9bd6-6bca401e2791-cert\") pod \"ingress-canary-hzhk4\" (UID: \"eea7d684-a24b-4464-9bd6-6bca401e2791\") " pod="openshift-ingress-canary/ingress-canary-hzhk4"
Jul 12 17:54:35.907224 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f14ee62-aa17-47d0-8e04-eb49058e0e91-data-volume\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.907224 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r29lc\" (UniqueName: \"kubernetes.io/projected/19eadb03-b4c4-4f60-a60e-db68e797d2ad-kube-api-access-r29lc\") pod \"dns-default-pmx65\" (UID: \"19eadb03-b4c4-4f60-a60e-db68e797d2ad\") " pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:35.907379 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.907379 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907330    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v7n5\" (UniqueName: \"kubernetes.io/projected/eea7d684-a24b-4464-9bd6-6bca401e2791-kube-api-access-5v7n5\") pod \"ingress-canary-hzhk4\" (UID: \"eea7d684-a24b-4464-9bd6-6bca401e2791\") " pod="openshift-ingress-canary/ingress-canary-hzhk4"
Jul 12 17:54:35.907379 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907352    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.907379 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:35.907366    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 17:54:35.907575 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fmkq\" (UniqueName: \"kubernetes.io/projected/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-api-access-5fmkq\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.907575 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:35.907430    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm podName:3f14ee62-aa17-47d0-8e04-eb49058e0e91 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:36.407415607 +0000 UTC m=+51.607535239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8xmkt" (UID: "3f14ee62-aa17-47d0-8e04-eb49058e0e91") : configmap "kube-rbac-proxy" not found
Jul 12 17:54:35.907575 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907447    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19eadb03-b4c4-4f60-a60e-db68e797d2ad-tmp-dir\") pod \"dns-default-pmx65\" (UID: \"19eadb03-b4c4-4f60-a60e-db68e797d2ad\") " pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:35.907575 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:35.907476    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 17:54:35.907575 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907478    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f14ee62-aa17-47d0-8e04-eb49058e0e91-crio-socket\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.907575 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907503    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19eadb03-b4c4-4f60-a60e-db68e797d2ad-config-volume\") pod \"dns-default-pmx65\" (UID: \"19eadb03-b4c4-4f60-a60e-db68e797d2ad\") " pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:35.907575 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:35.907546    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls podName:3f14ee62-aa17-47d0-8e04-eb49058e0e91 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:36.407506997 +0000 UTC m=+51.607626631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8xmkt" (UID: "3f14ee62-aa17-47d0-8e04-eb49058e0e91") : secret "insights-runtime-extractor-tls" not found
Jul 12 17:54:35.907575 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907571    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f14ee62-aa17-47d0-8e04-eb49058e0e91-data-volume\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.907852 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19eadb03-b4c4-4f60-a60e-db68e797d2ad-metrics-tls\") pod \"dns-default-pmx65\" (UID: \"19eadb03-b4c4-4f60-a60e-db68e797d2ad\") " pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:35.907852 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907742    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f14ee62-aa17-47d0-8e04-eb49058e0e91-crio-socket\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.907852 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.907745    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19eadb03-b4c4-4f60-a60e-db68e797d2ad-tmp-dir\") pod \"dns-default-pmx65\" (UID: \"19eadb03-b4c4-4f60-a60e-db68e797d2ad\") " pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:35.908025 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.908009    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19eadb03-b4c4-4f60-a60e-db68e797d2ad-config-volume\") pod \"dns-default-pmx65\" (UID: \"19eadb03-b4c4-4f60-a60e-db68e797d2ad\") " pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:35.911756 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.911730    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19eadb03-b4c4-4f60-a60e-db68e797d2ad-metrics-tls\") pod \"dns-default-pmx65\" (UID: \"19eadb03-b4c4-4f60-a60e-db68e797d2ad\") " pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:35.917455 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.917430    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r29lc\" (UniqueName: \"kubernetes.io/projected/19eadb03-b4c4-4f60-a60e-db68e797d2ad-kube-api-access-r29lc\") pod \"dns-default-pmx65\" (UID: \"19eadb03-b4c4-4f60-a60e-db68e797d2ad\") " pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:35.917631 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.917615    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fmkq\" (UniqueName: \"kubernetes.io/projected/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-api-access-5fmkq\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:35.994077 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:35.994056    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:36.008439 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:36.008420    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v7n5\" (UniqueName: \"kubernetes.io/projected/eea7d684-a24b-4464-9bd6-6bca401e2791-kube-api-access-5v7n5\") pod \"ingress-canary-hzhk4\" (UID: \"eea7d684-a24b-4464-9bd6-6bca401e2791\") " pod="openshift-ingress-canary/ingress-canary-hzhk4"
Jul 12 17:54:36.008507 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:36.008466    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eea7d684-a24b-4464-9bd6-6bca401e2791-cert\") pod \"ingress-canary-hzhk4\" (UID: \"eea7d684-a24b-4464-9bd6-6bca401e2791\") " pod="openshift-ingress-canary/ingress-canary-hzhk4"
Jul 12 17:54:36.010633 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:36.010616    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eea7d684-a24b-4464-9bd6-6bca401e2791-cert\") pod \"ingress-canary-hzhk4\" (UID: \"eea7d684-a24b-4464-9bd6-6bca401e2791\") " pod="openshift-ingress-canary/ingress-canary-hzhk4"
Jul 12 17:54:36.015867 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:36.015847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v7n5\" (UniqueName: \"kubernetes.io/projected/eea7d684-a24b-4464-9bd6-6bca401e2791-kube-api-access-5v7n5\") pod \"ingress-canary-hzhk4\" (UID: \"eea7d684-a24b-4464-9bd6-6bca401e2791\") " pod="openshift-ingress-canary/ingress-canary-hzhk4"
Jul 12 17:54:36.063998 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:36.063971    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-89mdh_734454b5-e121-4543-ab79-ecfe0d985f96/dns-node-resolver/0.log"
Jul 12 17:54:36.085221 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:36.085199    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hzhk4"
Jul 12 17:54:36.176000 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:36.175973    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pmx65"]
Jul 12 17:54:36.181145 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:36.181116    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19eadb03_b4c4_4f60_a60e_db68e797d2ad.slice/crio-e5ae44c7674b68902ba53f375ecc44312fa8b0b9e89a9d33b46542d5c19caa2c WatchSource:0}: Error finding container e5ae44c7674b68902ba53f375ecc44312fa8b0b9e89a9d33b46542d5c19caa2c: Status 404 returned error can't find the container with id e5ae44c7674b68902ba53f375ecc44312fa8b0b9e89a9d33b46542d5c19caa2c
Jul 12 17:54:36.218670 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:36.218648    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hzhk4"]
Jul 12 17:54:36.220960 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:36.220933    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeea7d684_a24b_4464_9bd6_6bca401e2791.slice/crio-bc02e467c2dd67e1c501ee8e5794adc15f17f53ffbf13011466d4b0856a79b3e WatchSource:0}: Error finding container bc02e467c2dd67e1c501ee8e5794adc15f17f53ffbf13011466d4b0856a79b3e: Status 404 returned error can't find the container with id bc02e467c2dd67e1c501ee8e5794adc15f17f53ffbf13011466d4b0856a79b3e
Jul 12 17:54:36.412027 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:36.412004    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:36.412027 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:36.412034    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:36.412457 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:36.412117    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 17:54:36.412457 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:36.412131    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 17:54:36.412457 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:36.412172    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm podName:3f14ee62-aa17-47d0-8e04-eb49058e0e91 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:37.41215568 +0000 UTC m=+52.612275313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8xmkt" (UID: "3f14ee62-aa17-47d0-8e04-eb49058e0e91") : configmap "kube-rbac-proxy" not found
Jul 12 17:54:36.412457 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:36.412185    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls podName:3f14ee62-aa17-47d0-8e04-eb49058e0e91 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:37.41217931 +0000 UTC m=+52.612298942 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8xmkt" (UID: "3f14ee62-aa17-47d0-8e04-eb49058e0e91") : secret "insights-runtime-extractor-tls" not found
Jul 12 17:54:36.753309 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:36.753251    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pmx65" event={"ID":"19eadb03-b4c4-4f60-a60e-db68e797d2ad","Type":"ContainerStarted","Data":"e5ae44c7674b68902ba53f375ecc44312fa8b0b9e89a9d33b46542d5c19caa2c"}
Jul 12 17:54:36.754602 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:36.754575    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hzhk4" event={"ID":"eea7d684-a24b-4464-9bd6-6bca401e2791","Type":"ContainerStarted","Data":"bc02e467c2dd67e1c501ee8e5794adc15f17f53ffbf13011466d4b0856a79b3e"}
Jul 12 17:54:37.319791 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:37.319760    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:37.319955 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:37.319760    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:37.319955 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:37.319760    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:37.323801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:37.323778    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 12 17:54:37.323801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:37.323790    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 12 17:54:37.323979 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:37.323826    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-84rf9\""
Jul 12 17:54:37.323979 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:37.323842    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 12 17:54:37.323979 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:37.323778    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 12 17:54:37.323979 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:37.323898    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h9zfm\""
Jul 12 17:54:37.419700 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:37.419666    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:37.420049 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:37.419750    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 17:54:37.420049 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:37.419801    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm podName:3f14ee62-aa17-47d0-8e04-eb49058e0e91 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:39.419788051 +0000 UTC m=+54.619907683 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8xmkt" (UID: "3f14ee62-aa17-47d0-8e04-eb49058e0e91") : configmap "kube-rbac-proxy" not found
Jul 12 17:54:37.420049 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:37.419852    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:37.420049 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:37.419975    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 17:54:37.420049 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:37.420030    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls podName:3f14ee62-aa17-47d0-8e04-eb49058e0e91 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:39.420011371 +0000 UTC m=+54.620131005 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8xmkt" (UID: "3f14ee62-aa17-47d0-8e04-eb49058e0e91") : secret "insights-runtime-extractor-tls" not found
Jul 12 17:54:37.439997 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:37.439978    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b2njf_b092918b-d322-40b1-b850-33b856bb1ba4/node-ca/0.log"
Jul 12 17:54:38.344756 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.344723    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jg6t7"]
Jul 12 17:54:38.363199 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.363176    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jg6t7"]
Jul 12 17:54:38.363341 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.363296    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:38.365976 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.365956    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 12 17:54:38.366120 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.366070    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 12 17:54:38.366803 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.366782    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 12 17:54:38.367018 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.366946    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 12 17:54:38.367018 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.367009    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-drstn\""
Jul 12 17:54:38.367166 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.367033    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 12 17:54:38.528084 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.528050    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3094dd13-57e9-445b-90e3-d22a097e4fec-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jg6t7\" (UID: \"3094dd13-57e9-445b-90e3-d22a097e4fec\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:38.528533 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.528101    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3094dd13-57e9-445b-90e3-d22a097e4fec-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jg6t7\" (UID: \"3094dd13-57e9-445b-90e3-d22a097e4fec\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:38.528533 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.528123    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jljwr\" (UniqueName: \"kubernetes.io/projected/3094dd13-57e9-445b-90e3-d22a097e4fec-kube-api-access-jljwr\") pod \"prometheus-operator-64785b6675-jg6t7\" (UID: \"3094dd13-57e9-445b-90e3-d22a097e4fec\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:38.528533 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.528236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3094dd13-57e9-445b-90e3-d22a097e4fec-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jg6t7\" (UID: \"3094dd13-57e9-445b-90e3-d22a097e4fec\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:38.629607 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.629530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3094dd13-57e9-445b-90e3-d22a097e4fec-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jg6t7\" (UID: \"3094dd13-57e9-445b-90e3-d22a097e4fec\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:38.629607 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.629575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3094dd13-57e9-445b-90e3-d22a097e4fec-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jg6t7\" (UID: \"3094dd13-57e9-445b-90e3-d22a097e4fec\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:38.629607 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.629592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jljwr\" (UniqueName: \"kubernetes.io/projected/3094dd13-57e9-445b-90e3-d22a097e4fec-kube-api-access-jljwr\") pod \"prometheus-operator-64785b6675-jg6t7\" (UID: \"3094dd13-57e9-445b-90e3-d22a097e4fec\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:38.629838 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.629630    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3094dd13-57e9-445b-90e3-d22a097e4fec-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jg6t7\" (UID: \"3094dd13-57e9-445b-90e3-d22a097e4fec\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:38.629838 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:38.629711    2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 12 17:54:38.629838 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:38.629782    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3094dd13-57e9-445b-90e3-d22a097e4fec-prometheus-operator-tls podName:3094dd13-57e9-445b-90e3-d22a097e4fec nodeName:}" failed. No retries permitted until 2026-07-12 17:54:39.129764169 +0000 UTC m=+54.329883802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3094dd13-57e9-445b-90e3-d22a097e4fec-prometheus-operator-tls") pod "prometheus-operator-64785b6675-jg6t7" (UID: "3094dd13-57e9-445b-90e3-d22a097e4fec") : secret "prometheus-operator-tls" not found
Jul 12 17:54:38.630507 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.630482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3094dd13-57e9-445b-90e3-d22a097e4fec-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jg6t7\" (UID: \"3094dd13-57e9-445b-90e3-d22a097e4fec\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:38.632796 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.632779    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3094dd13-57e9-445b-90e3-d22a097e4fec-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jg6t7\" (UID: \"3094dd13-57e9-445b-90e3-d22a097e4fec\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:38.638157 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:38.638140    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jljwr\" (UniqueName: \"kubernetes.io/projected/3094dd13-57e9-445b-90e3-d22a097e4fec-kube-api-access-jljwr\") pod \"prometheus-operator-64785b6675-jg6t7\" (UID: \"3094dd13-57e9-445b-90e3-d22a097e4fec\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:39.133582 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:39.133556    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3094dd13-57e9-445b-90e3-d22a097e4fec-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jg6t7\" (UID: \"3094dd13-57e9-445b-90e3-d22a097e4fec\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:39.136453 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:39.136403    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3094dd13-57e9-445b-90e3-d22a097e4fec-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jg6t7\" (UID: \"3094dd13-57e9-445b-90e3-d22a097e4fec\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:39.273639 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:39.273607    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7"
Jul 12 17:54:39.405200 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:39.404875    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jg6t7"]
Jul 12 17:54:39.407536 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:39.407496    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3094dd13_57e9_445b_90e3_d22a097e4fec.slice/crio-b3a4aa7c8b742104b35b7c0d09c867a3cc89937a39b941992ff5d69ff1b7ec73 WatchSource:0}: Error finding container b3a4aa7c8b742104b35b7c0d09c867a3cc89937a39b941992ff5d69ff1b7ec73: Status 404 returned error can't find the container with id b3a4aa7c8b742104b35b7c0d09c867a3cc89937a39b941992ff5d69ff1b7ec73
Jul 12 17:54:39.435319 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:39.435294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:39.435443 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:39.435426    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 17:54:39.435481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:39.435464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:39.435481 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:39.435477    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls podName:3f14ee62-aa17-47d0-8e04-eb49058e0e91 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:43.435462137 +0000 UTC m=+58.635581769 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8xmkt" (UID: "3f14ee62-aa17-47d0-8e04-eb49058e0e91") : secret "insights-runtime-extractor-tls" not found
Jul 12 17:54:39.435574 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:39.435513    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 17:54:39.435574 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:39.435572    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm podName:3f14ee62-aa17-47d0-8e04-eb49058e0e91 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:43.43556072 +0000 UTC m=+58.635680353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8xmkt" (UID: "3f14ee62-aa17-47d0-8e04-eb49058e0e91") : configmap "kube-rbac-proxy" not found
Jul 12 17:54:39.763544 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:39.763502    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hzhk4" event={"ID":"eea7d684-a24b-4464-9bd6-6bca401e2791","Type":"ContainerStarted","Data":"a0a8fd7fa7c14ed1924aba687e5c8336f64ba022499a7e9c499f27286a6cbb50"}
Jul 12 17:54:39.764702 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:39.764677    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7" event={"ID":"3094dd13-57e9-445b-90e3-d22a097e4fec","Type":"ContainerStarted","Data":"b3a4aa7c8b742104b35b7c0d09c867a3cc89937a39b941992ff5d69ff1b7ec73"}
Jul 12 17:54:39.766245 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:39.766225    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pmx65" event={"ID":"19eadb03-b4c4-4f60-a60e-db68e797d2ad","Type":"ContainerStarted","Data":"7a44502869a305bf2997c32802898c8d487b3aa0d776bba124e93fb0ea985859"}
Jul 12 17:54:39.766320 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:39.766271    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pmx65" event={"ID":"19eadb03-b4c4-4f60-a60e-db68e797d2ad","Type":"ContainerStarted","Data":"f53a3ada68e65bc2e75b9fee61e567e8e88b82802a680472b23bdb4f00ef7a90"}
Jul 12 17:54:39.766320 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:39.766310    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:39.777734 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:39.777700    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hzhk4" podStartSLOduration=1.996806004 podStartE2EDuration="4.777683474s" totalImagesPullingTime="2.78087747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:39.777388874 +0000 UTC m=+54.977508531" watchObservedRunningTime="2026-07-12 17:54:39.777683474 +0000 UTC m=+54.977803125"
Jul 12 17:54:39.793291 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:39.793243    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pmx65" podStartSLOduration=1.9789149350000002 podStartE2EDuration="4.793227791s" totalImagesPullingTime="2.814312856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:39.791848641 +0000 UTC m=+54.991968296" watchObservedRunningTime="2026-07-12 17:54:39.793227791 +0000 UTC m=+54.993347445"
Jul 12 17:54:40.772502 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:40.772457    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7" event={"ID":"3094dd13-57e9-445b-90e3-d22a097e4fec","Type":"ContainerStarted","Data":"d597067fdcff211c295b5900f9fd93346ecbead0f8a6ee06783e490d2c41d37c"}
Jul 12 17:54:40.773052 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:40.772539    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7" event={"ID":"3094dd13-57e9-445b-90e3-d22a097e4fec","Type":"ContainerStarted","Data":"cf9b0c9fd47b6f5ce9fcea32a539c26e1f5015de138c71a9f4aeb8b279a9d278"}
Jul 12 17:54:40.791283 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:40.791231    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-jg6t7" podStartSLOduration=1.593259308 podStartE2EDuration="2.791211962s" totalImagesPullingTime="1.197952654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:40.79087043 +0000 UTC m=+55.990990103" watchObservedRunningTime="2026-07-12 17:54:40.791211962 +0000 UTC m=+55.991331616"
Jul 12 17:54:42.489669 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.489638    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dfb658bb9-htl8r"]
Jul 12 17:54:42.493765 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.493742    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.496104 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.496084    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 12 17:54:42.497197 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.497177    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 12 17:54:42.497286 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.497235    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 12 17:54:42.497286 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.497254    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4rmzj\""
Jul 12 17:54:42.497286 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.497256    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 12 17:54:42.497286 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.497266    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 12 17:54:42.497490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.497388    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 12 17:54:42.497490 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.497391    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 12 17:54:42.501987 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.501966    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 12 17:54:42.505284 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.505265    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dfb658bb9-htl8r"]
Jul 12 17:54:42.657156 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.657125    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"]
Jul 12 17:54:42.657463 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.657444    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-trusted-ca-bundle\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.657566 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.657489    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-oauth-serving-cert\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.657566 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.657509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47f09a89-9355-4349-b2bb-b2ee714b737b-console-oauth-config\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.657566 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.657540    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-service-ca\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.657726 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.657616    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47f09a89-9355-4349-b2bb-b2ee714b737b-console-serving-cert\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.657726 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.657640    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2zvw\" (UniqueName: \"kubernetes.io/projected/47f09a89-9355-4349-b2bb-b2ee714b737b-kube-api-access-x2zvw\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.657726 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.657659    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-console-config\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.659264 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.659246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:42.661449 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.661430    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 12 17:54:42.661566 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.661438    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 12 17:54:42.661566 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.661505    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5h6s8\""
Jul 12 17:54:42.671362 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.671341    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"]
Jul 12 17:54:42.689025 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.689008    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"]
Jul 12 17:54:42.690956 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.690943    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.693284 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.693269    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 12 17:54:42.693284 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.693278    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 12 17:54:42.693533 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.693505    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-t66cc\""
Jul 12 17:54:42.693611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.693530    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 12 17:54:42.702434 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.702417    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"]
Jul 12 17:54:42.705562 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.705546    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7wtg8"]
Jul 12 17:54:42.707323 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.707309    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.709426 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.709403    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 12 17:54:42.709426 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.709404    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 12 17:54:42.709575 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.709456    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 12 17:54:42.709575 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.709481    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wl5tq\""
Jul 12 17:54:42.758998 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.758954    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47f09a89-9355-4349-b2bb-b2ee714b737b-console-serving-cert\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.758998 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.758980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2zvw\" (UniqueName: \"kubernetes.io/projected/47f09a89-9355-4349-b2bb-b2ee714b737b-kube-api-access-x2zvw\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.759125 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.758999    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-xc6lx\" (UID: \"f1b5a7de-cfef-4341-b852-add2f0d8ae2e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:42.759125 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.759016    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-console-config\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.759125 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.759035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn4hl\" (UniqueName: \"kubernetes.io/projected/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-kube-api-access-wn4hl\") pod \"openshift-state-metrics-84bd68b896-xc6lx\" (UID: \"f1b5a7de-cfef-4341-b852-add2f0d8ae2e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:42.759227 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.759189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-trusted-ca-bundle\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.759264 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.759230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-oauth-serving-cert\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.759264 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.759256    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-xc6lx\" (UID: \"f1b5a7de-cfef-4341-b852-add2f0d8ae2e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:42.759332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.759278    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47f09a89-9355-4349-b2bb-b2ee714b737b-console-oauth-config\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.759332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.759294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-service-ca\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.759332 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.759323    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-xc6lx\" (UID: \"f1b5a7de-cfef-4341-b852-add2f0d8ae2e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:42.759884 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.759854    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-console-config\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.759982 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.759922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-trusted-ca-bundle\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.759982 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.759936    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-oauth-serving-cert\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.759982 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.759969    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-service-ca\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.762639 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.762623    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47f09a89-9355-4349-b2bb-b2ee714b737b-console-serving-cert\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.762639 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.762633    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47f09a89-9355-4349-b2bb-b2ee714b737b-console-oauth-config\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.766094 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.766071    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2zvw\" (UniqueName: \"kubernetes.io/projected/47f09a89-9355-4349-b2bb-b2ee714b737b-kube-api-access-x2zvw\") pod \"console-5dfb658bb9-htl8r\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") " pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.803278 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.803261    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:42.860775 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.860738    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-wtmp\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.860915 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.860787    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7e4dbf6-2a10-4a43-8bac-d745892049dd-sys\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.860915 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.860830    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.860915 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.860872    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn4hl\" (UniqueName: \"kubernetes.io/projected/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-kube-api-access-wn4hl\") pod \"openshift-state-metrics-84bd68b896-xc6lx\" (UID: \"f1b5a7de-cfef-4341-b852-add2f0d8ae2e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:42.860915 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.860895    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxvdp\" (UniqueName: \"kubernetes.io/projected/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-api-access-jxvdp\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.861131 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.860962    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-tls\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.861131 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.860982    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvsgt\" (UniqueName: \"kubernetes.io/projected/f7e4dbf6-2a10-4a43-8bac-d745892049dd-kube-api-access-vvsgt\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.861131 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.861004    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7e4dbf6-2a10-4a43-8bac-d745892049dd-metrics-client-ca\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.861131 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.861031    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-xc6lx\" (UID: \"f1b5a7de-cfef-4341-b852-add2f0d8ae2e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:42.861131 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.861053    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f7e4dbf6-2a10-4a43-8bac-d745892049dd-root\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.861131 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.861091    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-xc6lx\" (UID: \"f1b5a7de-cfef-4341-b852-add2f0d8ae2e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:42.861131 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.861114    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.862005 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.861144    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-accelerators-collector-config\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.862005 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.861171    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-xc6lx\" (UID: \"f1b5a7de-cfef-4341-b852-add2f0d8ae2e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:42.862005 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.861284    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-textfile\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.862005 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.861464    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.862005 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.861571    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.862005 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.861600    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.862005 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.861632    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.862005 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:42.861371    2578 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 12 17:54:42.862005 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:42.861776    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-openshift-state-metrics-tls podName:f1b5a7de-cfef-4341-b852-add2f0d8ae2e nodeName:}" failed. No retries permitted until 2026-07-12 17:54:43.361756891 +0000 UTC m=+58.561876537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-xc6lx" (UID: "f1b5a7de-cfef-4341-b852-add2f0d8ae2e") : secret "openshift-state-metrics-tls" not found
Jul 12 17:54:42.862005 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.861897    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-xc6lx\" (UID: \"f1b5a7de-cfef-4341-b852-add2f0d8ae2e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:42.864197 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.864173    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-xc6lx\" (UID: \"f1b5a7de-cfef-4341-b852-add2f0d8ae2e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:42.873949 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.873929    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn4hl\" (UniqueName: \"kubernetes.io/projected/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-kube-api-access-wn4hl\") pod \"openshift-state-metrics-84bd68b896-xc6lx\" (UID: \"f1b5a7de-cfef-4341-b852-add2f0d8ae2e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:42.941100 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.941073    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dfb658bb9-htl8r"]
Jul 12 17:54:42.944084 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:42.944050    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f09a89_9355_4349_b2bb_b2ee714b737b.slice/crio-43a51e638dae9870c296979496de83c919ce4346226b3ddec6f3b1ecfbb1a0f0 WatchSource:0}: Error finding container 43a51e638dae9870c296979496de83c919ce4346226b3ddec6f3b1ecfbb1a0f0: Status 404 returned error can't find the container with id 43a51e638dae9870c296979496de83c919ce4346226b3ddec6f3b1ecfbb1a0f0
Jul 12 17:54:42.962341 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-tls\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.962428 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvsgt\" (UniqueName: \"kubernetes.io/projected/f7e4dbf6-2a10-4a43-8bac-d745892049dd-kube-api-access-vvsgt\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.962428 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962363    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7e4dbf6-2a10-4a43-8bac-d745892049dd-metrics-client-ca\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.962428 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962379    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f7e4dbf6-2a10-4a43-8bac-d745892049dd-root\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.962428 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.962586 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962456    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f7e4dbf6-2a10-4a43-8bac-d745892049dd-root\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.962586 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:42.962488    2578 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 12 17:54:42.962586 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962532    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-accelerators-collector-config\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.962586 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:42.962563    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-state-metrics-tls podName:59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd nodeName:}" failed. No retries permitted until 2026-07-12 17:54:43.462544079 +0000 UTC m=+58.662663711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-b2nhm" (UID: "59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd") : secret "kube-state-metrics-tls" not found
Jul 12 17:54:42.962729 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-textfile\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.962729 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962650    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.962729 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.962729 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962721    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.962976 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962748    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.962976 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962801    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-wtmp\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.962976 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7e4dbf6-2a10-4a43-8bac-d745892049dd-sys\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.962976 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.962976 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.962893    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxvdp\" (UniqueName: \"kubernetes.io/projected/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-api-access-jxvdp\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.963217 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.963005    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-accelerators-collector-config\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.963217 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.963011    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7e4dbf6-2a10-4a43-8bac-d745892049dd-metrics-client-ca\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.963217 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.963129    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-wtmp\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.963217 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.963128    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7e4dbf6-2a10-4a43-8bac-d745892049dd-sys\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.963407 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.963345    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-textfile\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.963407 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.963364    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.963508 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.963416    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.963508 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.963444    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.964913 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.964883    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-tls\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.965057 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.965039    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7e4dbf6-2a10-4a43-8bac-d745892049dd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.965305 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.965289    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:42.970428 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.970410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvsgt\" (UniqueName: \"kubernetes.io/projected/f7e4dbf6-2a10-4a43-8bac-d745892049dd-kube-api-access-vvsgt\") pod \"node-exporter-7wtg8\" (UID: \"f7e4dbf6-2a10-4a43-8bac-d745892049dd\") " pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:42.970767 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:42.970749    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxvdp\" (UniqueName: \"kubernetes.io/projected/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-api-access-jxvdp\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:43.014839 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.014794    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7wtg8"
Jul 12 17:54:43.023538 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:43.023496    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e4dbf6_2a10_4a43_8bac_d745892049dd.slice/crio-fe4929f0d0885d11f4732927b5b06f4db364a3eff734fe2d684bf2eae0d91cec WatchSource:0}: Error finding container fe4929f0d0885d11f4732927b5b06f4db364a3eff734fe2d684bf2eae0d91cec: Status 404 returned error can't find the container with id fe4929f0d0885d11f4732927b5b06f4db364a3eff734fe2d684bf2eae0d91cec
Jul 12 17:54:43.367226 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.367161    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-xc6lx\" (UID: \"f1b5a7de-cfef-4341-b852-add2f0d8ae2e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:43.369261 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.369229    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1b5a7de-cfef-4341-b852-add2f0d8ae2e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-xc6lx\" (UID: \"f1b5a7de-cfef-4341-b852-add2f0d8ae2e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:43.468448 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.468419    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:43.468619 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.468534    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:43.468619 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:43.468601    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 12 17:54:43.468729 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.468656    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:43.468729 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:43.468672    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls podName:3f14ee62-aa17-47d0-8e04-eb49058e0e91 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:51.468651483 +0000 UTC m=+66.668771118 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8xmkt" (UID: "3f14ee62-aa17-47d0-8e04-eb49058e0e91") : secret "insights-runtime-extractor-tls" not found
Jul 12 17:54:43.468729 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:43.468721    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 12 17:54:43.468854 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:43.468772    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm podName:3f14ee62-aa17-47d0-8e04-eb49058e0e91 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:51.468756124 +0000 UTC m=+66.668875780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8xmkt" (UID: "3f14ee62-aa17-47d0-8e04-eb49058e0e91") : configmap "kube-rbac-proxy" not found
Jul 12 17:54:43.470778 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.470751    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-b2nhm\" (UID: \"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:43.567935 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.567889    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"
Jul 12 17:54:43.598913 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.598886    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"
Jul 12 17:54:43.732312 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.732280    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx"]
Jul 12 17:54:43.735926 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.735664    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 17:54:43.736958 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:43.736922    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b5a7de_cfef_4341_b852_add2f0d8ae2e.slice/crio-2e7aef1d53a1d630adf824ba748e12049b1f15560ac03c48c9a7d7745173042b WatchSource:0}: Error finding container 2e7aef1d53a1d630adf824ba748e12049b1f15560ac03c48c9a7d7745173042b: Status 404 returned error can't find the container with id 2e7aef1d53a1d630adf824ba748e12049b1f15560ac03c48c9a7d7745173042b
Jul 12 17:54:43.743417 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.743313    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.746992 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.745705    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 12 17:54:43.746992 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.745788    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t22md\""
Jul 12 17:54:43.746992 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.745904    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 12 17:54:43.746992 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.745943    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 12 17:54:43.746992 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.745970    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 12 17:54:43.746992 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.745946    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 12 17:54:43.746992 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.746104    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 12 17:54:43.746992 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.746777    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 12 17:54:43.746992 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.746863    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 12 17:54:43.747538 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.747038    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 12 17:54:43.751895 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.751872    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 17:54:43.767402 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.767377    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-b2nhm"]
Jul 12 17:54:43.772807 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:43.772578    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59e19bd9_7fdb_49d9_b2f9_ff446e4c4dcd.slice/crio-4d4038c9ca650591d85f3f79e0a6f8f10d00829b96e70a6b1d61421e337368cb WatchSource:0}: Error finding container 4d4038c9ca650591d85f3f79e0a6f8f10d00829b96e70a6b1d61421e337368cb: Status 404 returned error can't find the container with id 4d4038c9ca650591d85f3f79e0a6f8f10d00829b96e70a6b1d61421e337368cb
Jul 12 17:54:43.785426 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.785359    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfb658bb9-htl8r" event={"ID":"47f09a89-9355-4349-b2bb-b2ee714b737b","Type":"ContainerStarted","Data":"43a51e638dae9870c296979496de83c919ce4346226b3ddec6f3b1ecfbb1a0f0"}
Jul 12 17:54:43.787003 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.786943    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm" event={"ID":"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd","Type":"ContainerStarted","Data":"4d4038c9ca650591d85f3f79e0a6f8f10d00829b96e70a6b1d61421e337368cb"}
Jul 12 17:54:43.788288 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.788226    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx" event={"ID":"f1b5a7de-cfef-4341-b852-add2f0d8ae2e","Type":"ContainerStarted","Data":"2e7aef1d53a1d630adf824ba748e12049b1f15560ac03c48c9a7d7745173042b"}
Jul 12 17:54:43.789755 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.789733    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7wtg8" event={"ID":"f7e4dbf6-2a10-4a43-8bac-d745892049dd","Type":"ContainerStarted","Data":"fe4929f0d0885d11f4732927b5b06f4db364a3eff734fe2d684bf2eae0d91cec"}
Jul 12 17:54:43.873309 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.873183    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.873309 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.873230    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzllh\" (UniqueName: \"kubernetes.io/projected/6140b6d8-80cb-4a67-834f-4a348801f202-kube-api-access-wzllh\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.873309 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.873255    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-web-config\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.873309 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.873281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.873309 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.873304    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.873768 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.873354    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.873768 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.873435    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-config-volume\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.873768 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.873474    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.873768 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.873498    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.873768 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.873565    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6140b6d8-80cb-4a67-834f-4a348801f202-config-out\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.873768 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.873616    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6140b6d8-80cb-4a67-834f-4a348801f202-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.873768 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.873667    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.873768 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.873715    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.975097 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.975060    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.975256 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.975111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.975256 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.975140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6140b6d8-80cb-4a67-834f-4a348801f202-config-out\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.975256 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.975243    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6140b6d8-80cb-4a67-834f-4a348801f202-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.975401 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.975311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.975401 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.975357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.975401 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.975386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.975571 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.975414    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzllh\" (UniqueName: \"kubernetes.io/projected/6140b6d8-80cb-4a67-834f-4a348801f202-kube-api-access-wzllh\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.975571 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.975440    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-web-config\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.975571 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.975463    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.975571 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.975486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.975571 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.975554    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.975797 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.975627    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-config-volume\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.977454 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.976295    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.977454 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.976846    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.977454 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:43.976963    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-trusted-ca-bundle podName:6140b6d8-80cb-4a67-834f-4a348801f202 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:44.476937512 +0000 UTC m=+59.677057166 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202") : configmap references non-existent config key: ca-bundle.crt
Jul 12 17:54:43.980015 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.979990    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.980578 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.980555    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.981193 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.981124    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.981312 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.981293    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-config-volume\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.981361 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.981340    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.981498 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.981474    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6140b6d8-80cb-4a67-834f-4a348801f202-config-out\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.982050 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.982030    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-web-config\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.982811 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.982463    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6140b6d8-80cb-4a67-834f-4a348801f202-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.983343 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.983321    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:43.984992 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:43.984943    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzllh\" (UniqueName: \"kubernetes.io/projected/6140b6d8-80cb-4a67-834f-4a348801f202-kube-api-access-wzllh\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:44.479791 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:44.479757    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:44.480767 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:44.480731    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:44.656463 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:44.656393    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:54:44.804535 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:44.804467    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx" event={"ID":"f1b5a7de-cfef-4341-b852-add2f0d8ae2e","Type":"ContainerStarted","Data":"ff9b3ab23de42c889f0d658691dcc9d7756814881001e63391de17e6d98e2f4b"}
Jul 12 17:54:44.804687 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:44.804602    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx" event={"ID":"f1b5a7de-cfef-4341-b852-add2f0d8ae2e","Type":"ContainerStarted","Data":"605d8a6cd3d13a7a02f0c6c61cd09d4a5c432657e7a69e7560b49c1d6090a225"}
Jul 12 17:54:44.806596 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:44.806570    2578 generic.go:356] "Generic (PLEG): container finished" podID="f7e4dbf6-2a10-4a43-8bac-d745892049dd" containerID="88bf770c6dcaf35106bd07911af3c441020b966b81a1b302cb3a60721fef66f7" exitCode=0
Jul 12 17:54:44.806707 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:44.806607    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7wtg8" event={"ID":"f7e4dbf6-2a10-4a43-8bac-d745892049dd","Type":"ContainerDied","Data":"88bf770c6dcaf35106bd07911af3c441020b966b81a1b302cb3a60721fef66f7"}
Jul 12 17:54:44.812914 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:44.812895    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 17:54:45.038335 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:45.038266    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6140b6d8_80cb_4a67_834f_4a348801f202.slice/crio-241893933a65c0f5cab2cdcb78b644b786af9d16d5130491dd2c29278ea8a3e6 WatchSource:0}: Error finding container 241893933a65c0f5cab2cdcb78b644b786af9d16d5130491dd2c29278ea8a3e6: Status 404 returned error can't find the container with id 241893933a65c0f5cab2cdcb78b644b786af9d16d5130491dd2c29278ea8a3e6
Jul 12 17:54:45.663226 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.663195    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"]
Jul 12 17:54:45.667438 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.667416    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.669807 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.669782    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 12 17:54:45.669905 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.669841    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 12 17:54:45.670227 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.670210    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 12 17:54:45.670466 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.670451    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 12 17:54:45.670577 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.670452    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 12 17:54:45.670829 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.670809    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lc25w\""
Jul 12 17:54:45.670909 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.670895    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-10n6skl6jnjin\""
Jul 12 17:54:45.678717 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.678692    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"]
Jul 12 17:54:45.795248 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.793083    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-grpc-tls\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.795248 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.793173    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4cmq\" (UniqueName: \"kubernetes.io/projected/8bb373d7-9362-4040-a204-6fdad64d4d8a-kube-api-access-j4cmq\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.795248 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.793225    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.795248 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.793324    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-tls\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.795248 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.793353    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.795248 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.793402    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.795248 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.793494    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bb373d7-9362-4040-a204-6fdad64d4d8a-metrics-client-ca\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.795248 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.793608    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.810647 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.810612    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerStarted","Data":"241893933a65c0f5cab2cdcb78b644b786af9d16d5130491dd2c29278ea8a3e6"}
Jul 12 17:54:45.894334 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.894294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bb373d7-9362-4040-a204-6fdad64d4d8a-metrics-client-ca\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.894611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.894388    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.894611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.894442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-grpc-tls\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.894611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.894481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4cmq\" (UniqueName: \"kubernetes.io/projected/8bb373d7-9362-4040-a204-6fdad64d4d8a-kube-api-access-j4cmq\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.894611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.894528    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.894611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.894557    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-tls\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.894611 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.894585    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.894957 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.894633    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.895150 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.895100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bb373d7-9362-4040-a204-6fdad64d4d8a-metrics-client-ca\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.898197 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.898137    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-grpc-tls\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.898973 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.898739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.898973 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.898791    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.899201 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.899180    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.899265 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.899236    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.899336 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.899278    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8bb373d7-9362-4040-a204-6fdad64d4d8a-secret-thanos-querier-tls\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.902938 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.902919    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4cmq\" (UniqueName: \"kubernetes.io/projected/8bb373d7-9362-4040-a204-6fdad64d4d8a-kube-api-access-j4cmq\") pod \"thanos-querier-b5cd8fc4c-6gtdm\" (UID: \"8bb373d7-9362-4040-a204-6fdad64d4d8a\") " pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:45.979557 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:45.979502    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:46.715770 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:46.715744    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-trbvl"
Jul 12 17:54:46.979636 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:46.979541    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"]
Jul 12 17:54:47.007331 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.007102    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"]
Jul 12 17:54:47.007331 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.007233    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.010152 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.010128    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 12 17:54:47.012222 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.011677    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 12 17:54:47.012222 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.011906    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-r4xrv\""
Jul 12 17:54:47.012222 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.011931    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 12 17:54:47.012222 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.012089    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 12 17:54:47.012222 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.012206    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ahahe38asqch\""
Jul 12 17:54:47.049960 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.049873    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"]
Jul 12 17:54:47.053349 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:47.053313    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bb373d7_9362_4040_a204_6fdad64d4d8a.slice/crio-c746689660cffaa6b258f02ac64074b72e9c7fc7058d84a2a3896918ceab0689 WatchSource:0}: Error finding container c746689660cffaa6b258f02ac64074b72e9c7fc7058d84a2a3896918ceab0689: Status 404 returned error can't find the container with id c746689660cffaa6b258f02ac64074b72e9c7fc7058d84a2a3896918ceab0689
Jul 12 17:54:47.207190 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.206811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4140ff72-996c-40d9-b2f6-40e241f6894e-client-ca-bundle\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.207190 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.206859    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsw7w\" (UniqueName: \"kubernetes.io/projected/4140ff72-996c-40d9-b2f6-40e241f6894e-kube-api-access-xsw7w\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.207190 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.206889    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4140ff72-996c-40d9-b2f6-40e241f6894e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.207190 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.206917    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4140ff72-996c-40d9-b2f6-40e241f6894e-metrics-server-audit-profiles\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.207190 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.207023    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4140ff72-996c-40d9-b2f6-40e241f6894e-secret-metrics-server-client-certs\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.207190 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.207047    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4140ff72-996c-40d9-b2f6-40e241f6894e-audit-log\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.207190 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.207085    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4140ff72-996c-40d9-b2f6-40e241f6894e-secret-metrics-server-tls\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.307907 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.307870    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4140ff72-996c-40d9-b2f6-40e241f6894e-secret-metrics-server-client-certs\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.308006 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.307908    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4140ff72-996c-40d9-b2f6-40e241f6894e-audit-log\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.308006 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.307951    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4140ff72-996c-40d9-b2f6-40e241f6894e-secret-metrics-server-tls\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.308137 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.308022    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4140ff72-996c-40d9-b2f6-40e241f6894e-client-ca-bundle\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.308137 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.308055    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsw7w\" (UniqueName: \"kubernetes.io/projected/4140ff72-996c-40d9-b2f6-40e241f6894e-kube-api-access-xsw7w\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.308137 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.308082    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4140ff72-996c-40d9-b2f6-40e241f6894e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.308137 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.308121    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4140ff72-996c-40d9-b2f6-40e241f6894e-metrics-server-audit-profiles\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.309209 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.309130    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4140ff72-996c-40d9-b2f6-40e241f6894e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.309304 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.309235    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4140ff72-996c-40d9-b2f6-40e241f6894e-audit-log\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.309734 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.309716    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4140ff72-996c-40d9-b2f6-40e241f6894e-metrics-server-audit-profiles\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.310878 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.310841    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4140ff72-996c-40d9-b2f6-40e241f6894e-secret-metrics-server-tls\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.311212 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.311154    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4140ff72-996c-40d9-b2f6-40e241f6894e-secret-metrics-server-client-certs\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.311307 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.311267    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4140ff72-996c-40d9-b2f6-40e241f6894e-client-ca-bundle\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.316550 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.316533    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsw7w\" (UniqueName: \"kubernetes.io/projected/4140ff72-996c-40d9-b2f6-40e241f6894e-kube-api-access-xsw7w\") pod \"metrics-server-6b65dd6d44-rcs4v\" (UID: \"4140ff72-996c-40d9-b2f6-40e241f6894e\") " pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.341917 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.341897    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:54:47.469117 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.469087    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"]
Jul 12 17:54:47.496580 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.496484    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dfb658bb9-htl8r"]
Jul 12 17:54:47.496580 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.496572    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"]
Jul 12 17:54:47.496849 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.496669    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"
Jul 12 17:54:47.499262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.499202    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8r8f2\""
Jul 12 17:54:47.499262 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.499249    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 12 17:54:47.506212 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.506182    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7ff4489496-rqmjp"]
Jul 12 17:54:47.511185 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.510630    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9jw2\" (UniqueName: \"kubernetes.io/projected/3b6d4c90-9b6b-4773-b866-88492d05373b-kube-api-access-h9jw2\") pod \"monitoring-plugin-5f68df7f58-tp7vq\" (UID: \"3b6d4c90-9b6b-4773-b866-88492d05373b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"
Jul 12 17:54:47.511185 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.510824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b6d4c90-9b6b-4773-b866-88492d05373b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-tp7vq\" (UID: \"3b6d4c90-9b6b-4773-b866-88492d05373b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"
Jul 12 17:54:47.533659 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.533630    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ff4489496-rqmjp"]
Jul 12 17:54:47.533807 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.533759    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.611891 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.611854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9jw2\" (UniqueName: \"kubernetes.io/projected/3b6d4c90-9b6b-4773-b866-88492d05373b-kube-api-access-h9jw2\") pod \"monitoring-plugin-5f68df7f58-tp7vq\" (UID: \"3b6d4c90-9b6b-4773-b866-88492d05373b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"
Jul 12 17:54:47.612042 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.611907    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-trusted-ca-bundle\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.612119 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.612035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6876w\" (UniqueName: \"kubernetes.io/projected/87079739-c667-438f-8d31-9e674f1002d9-kube-api-access-6876w\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.612119 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.612089    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-oauth-serving-cert\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.612218 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.612196    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87079739-c667-438f-8d31-9e674f1002d9-console-oauth-config\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.612333 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.612315    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b6d4c90-9b6b-4773-b866-88492d05373b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-tp7vq\" (UID: \"3b6d4c90-9b6b-4773-b866-88492d05373b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"
Jul 12 17:54:47.612395 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.612379    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-console-config\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.612443 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.612421    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-service-ca\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.612495 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.612451    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87079739-c667-438f-8d31-9e674f1002d9-console-serving-cert\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.612495 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:47.612472    2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 12 17:54:47.612686 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:54:47.612667    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6d4c90-9b6b-4773-b866-88492d05373b-monitoring-plugin-cert podName:3b6d4c90-9b6b-4773-b866-88492d05373b nodeName:}" failed. No retries permitted until 2026-07-12 17:54:48.112649821 +0000 UTC m=+63.312769456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/3b6d4c90-9b6b-4773-b866-88492d05373b-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-tp7vq" (UID: "3b6d4c90-9b6b-4773-b866-88492d05373b") : secret "monitoring-plugin-cert" not found
Jul 12 17:54:47.623021 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.622999    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9jw2\" (UniqueName: \"kubernetes.io/projected/3b6d4c90-9b6b-4773-b866-88492d05373b-kube-api-access-h9jw2\") pod \"monitoring-plugin-5f68df7f58-tp7vq\" (UID: \"3b6d4c90-9b6b-4773-b866-88492d05373b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"
Jul 12 17:54:47.713049 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.712988    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-service-ca\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.713049 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.713034    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87079739-c667-438f-8d31-9e674f1002d9-console-serving-cert\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.713343 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.713069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-trusted-ca-bundle\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.713343 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.713092    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6876w\" (UniqueName: \"kubernetes.io/projected/87079739-c667-438f-8d31-9e674f1002d9-kube-api-access-6876w\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.713343 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.713121    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-oauth-serving-cert\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.713343 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.713200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87079739-c667-438f-8d31-9e674f1002d9-console-oauth-config\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.713343 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.713341    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-console-config\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.714172 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.713838    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-service-ca\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.714172 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.714031    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-console-config\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.714172 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.714039    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-trusted-ca-bundle\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.714371 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.714262    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-oauth-serving-cert\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.715554 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.715482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87079739-c667-438f-8d31-9e674f1002d9-console-serving-cert\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.716694 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.716377    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87079739-c667-438f-8d31-9e674f1002d9-console-oauth-config\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.722880 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.722839    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6876w\" (UniqueName: \"kubernetes.io/projected/87079739-c667-438f-8d31-9e674f1002d9-kube-api-access-6876w\") pod \"console-7ff4489496-rqmjp\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") " pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.748297 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.748272    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"]
Jul 12 17:54:47.804246 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:47.804223    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4140ff72_996c_40d9_b2f6_40e241f6894e.slice/crio-0110dddc6b32587ff6cc055e9f4860c39ebe804bd64e557eecacf8e994681144 WatchSource:0}: Error finding container 0110dddc6b32587ff6cc055e9f4860c39ebe804bd64e557eecacf8e994681144: Status 404 returned error can't find the container with id 0110dddc6b32587ff6cc055e9f4860c39ebe804bd64e557eecacf8e994681144
Jul 12 17:54:47.819969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.819743    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm" event={"ID":"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd","Type":"ContainerStarted","Data":"c0da815732b53523c96cb564162b28f1019941359770e3548dc376053e08e241"}
Jul 12 17:54:47.819969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.819802    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm" event={"ID":"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd","Type":"ContainerStarted","Data":"0ff39ac9313872859764c25ab774bf46f1200ffcbc3a3b33ba0f5b78781f177e"}
Jul 12 17:54:47.819969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.819829    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm" event={"ID":"59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd","Type":"ContainerStarted","Data":"66d0b2f9ff7eb950f1da01cf6d06e70fb685f3305e7c76ddff0e8e0d2793598b"}
Jul 12 17:54:47.821078 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.821052    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm" event={"ID":"8bb373d7-9362-4040-a204-6fdad64d4d8a","Type":"ContainerStarted","Data":"c746689660cffaa6b258f02ac64074b72e9c7fc7058d84a2a3896918ceab0689"}
Jul 12 17:54:47.823470 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.823429    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx" event={"ID":"f1b5a7de-cfef-4341-b852-add2f0d8ae2e","Type":"ContainerStarted","Data":"51ee1df54522e7679381657780c8eade8765ac2fdc04595beaa3ebb1606f2dd5"}
Jul 12 17:54:47.825798 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.825776    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7wtg8" event={"ID":"f7e4dbf6-2a10-4a43-8bac-d745892049dd","Type":"ContainerStarted","Data":"8c9b7767e10041cb3324a90cd2db788afcb3788739b11386acab4b6f53488eb7"}
Jul 12 17:54:47.825888 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.825820    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7wtg8" event={"ID":"f7e4dbf6-2a10-4a43-8bac-d745892049dd","Type":"ContainerStarted","Data":"6a25509ddb940c9879f5beda9e4dc3a72a8579d59ff3e8de30f75e9eb796d1d3"}
Jul 12 17:54:47.827032 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.827010    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v" event={"ID":"4140ff72-996c-40d9-b2f6-40e241f6894e","Type":"ContainerStarted","Data":"0110dddc6b32587ff6cc055e9f4860c39ebe804bd64e557eecacf8e994681144"}
Jul 12 17:54:47.828743 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.828718    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfb658bb9-htl8r" event={"ID":"47f09a89-9355-4349-b2bb-b2ee714b737b","Type":"ContainerStarted","Data":"29b25b098f8be389bc00ece6ed1fa270f694e47264c07d77ec67d273f803aa52"}
Jul 12 17:54:47.830404 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.830370    2578 generic.go:356] "Generic (PLEG): container finished" podID="6140b6d8-80cb-4a67-834f-4a348801f202" containerID="1607fc4b4c86f6119f083d2bae32f8fe7999c599b59b92caea5f25f044ecb51a" exitCode=0
Jul 12 17:54:47.830488 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.830411    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerDied","Data":"1607fc4b4c86f6119f083d2bae32f8fe7999c599b59b92caea5f25f044ecb51a"}
Jul 12 17:54:47.839148 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.839105    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-b2nhm" podStartSLOduration=2.707439761 podStartE2EDuration="5.83909417s" totalImagesPullingTime="3.131654409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:47.837423969 +0000 UTC m=+63.037543640" watchObservedRunningTime="2026-07-12 17:54:47.83909417 +0000 UTC m=+63.039213832"
Jul 12 17:54:47.844392 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.844374    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:47.854860 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.854823    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7wtg8" podStartSLOduration=4.727898034 podStartE2EDuration="5.854809722s" totalImagesPullingTime="1.057264688s" totalInitContainerRuntime="69.647ms" isStatefulPod=true podCreationTimestamp="2026-07-12 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:47.853304913 +0000 UTC m=+63.053424567" watchObservedRunningTime="2026-07-12 17:54:47.854809722 +0000 UTC m=+63.054929354"
Jul 12 17:54:47.894307 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.894256    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xc6lx" podStartSLOduration=3.096981512 podStartE2EDuration="5.894238784s" totalImagesPullingTime="2.797257272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:47.893649226 +0000 UTC m=+63.093768881" watchObservedRunningTime="2026-07-12 17:54:47.894238784 +0000 UTC m=+63.094358441"
Jul 12 17:54:47.915677 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.915624    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dfb658bb9-htl8r" podStartSLOduration=1.954960813 podStartE2EDuration="5.915607312s" totalImagesPullingTime="3.960646499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:47.914768552 +0000 UTC m=+63.114888268" watchObservedRunningTime="2026-07-12 17:54:47.915607312 +0000 UTC m=+63.115726967"
Jul 12 17:54:47.982965 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:47.982928    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ff4489496-rqmjp"]
Jul 12 17:54:47.987213 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:47.987182    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87079739_c667_438f_8d31_9e674f1002d9.slice/crio-8637b3e3966a00b93f441f97b12dfb1e22d12fb3ab0901799b50bcddeb694906 WatchSource:0}: Error finding container 8637b3e3966a00b93f441f97b12dfb1e22d12fb3ab0901799b50bcddeb694906: Status 404 returned error can't find the container with id 8637b3e3966a00b93f441f97b12dfb1e22d12fb3ab0901799b50bcddeb694906
Jul 12 17:54:48.119413 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.118546    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b6d4c90-9b6b-4773-b866-88492d05373b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-tp7vq\" (UID: \"3b6d4c90-9b6b-4773-b866-88492d05373b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"
Jul 12 17:54:48.121590 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.121562    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b6d4c90-9b6b-4773-b866-88492d05373b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-tp7vq\" (UID: \"3b6d4c90-9b6b-4773-b866-88492d05373b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"
Jul 12 17:54:48.407856 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.407771    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"
Jul 12 17:54:48.554202 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.554034    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"]
Jul 12 17:54:48.557388 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:48.557358    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b6d4c90_9b6b_4773_b866_88492d05373b.slice/crio-0390ca57c3f2b1546b08faa1a14b5adaad26acbc1cc91949df94bf07671e5b38 WatchSource:0}: Error finding container 0390ca57c3f2b1546b08faa1a14b5adaad26acbc1cc91949df94bf07671e5b38: Status 404 returned error can't find the container with id 0390ca57c3f2b1546b08faa1a14b5adaad26acbc1cc91949df94bf07671e5b38
Jul 12 17:54:48.838051 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.838013    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq" event={"ID":"3b6d4c90-9b6b-4773-b866-88492d05373b","Type":"ContainerStarted","Data":"0390ca57c3f2b1546b08faa1a14b5adaad26acbc1cc91949df94bf07671e5b38"}
Jul 12 17:54:48.842770 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.842256    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff4489496-rqmjp" event={"ID":"87079739-c667-438f-8d31-9e674f1002d9","Type":"ContainerStarted","Data":"4fd80aa17de73108e42e5d1cb71241dfc22dc8b551b6cf3ddf3ad382666e318b"}
Jul 12 17:54:48.842770 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.842304    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff4489496-rqmjp" event={"ID":"87079739-c667-438f-8d31-9e674f1002d9","Type":"ContainerStarted","Data":"8637b3e3966a00b93f441f97b12dfb1e22d12fb3ab0901799b50bcddeb694906"}
Jul 12 17:54:48.858967 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.858563    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7ff4489496-rqmjp" podStartSLOduration=1.858547219 podStartE2EDuration="1.858547219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:48.858432271 +0000 UTC m=+64.058551926" watchObservedRunningTime="2026-07-12 17:54:48.858547219 +0000 UTC m=+64.058666871"
Jul 12 17:54:48.863798 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.863466    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 17:54:48.874107 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.874082    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.878066 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.877877    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 12 17:54:48.878698 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.878318    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 12 17:54:48.878698 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.878405    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 12 17:54:48.878698 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.878508    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 12 17:54:48.878978 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.878835    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 12 17:54:48.879480 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.879177    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bpsr8\""
Jul 12 17:54:48.879480 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.879293    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 12 17:54:48.880446 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.879754    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 12 17:54:48.880446 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.880048    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 12 17:54:48.880446 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.880237    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 12 17:54:48.882578 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.880899    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6bfscmt615doe\""
Jul 12 17:54:48.884027 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.883017    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 12 17:54:48.884027 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.883326    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 12 17:54:48.884027 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.883545    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 12 17:54:48.884027 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.883701    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 12 17:54:48.884027 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.883794    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 12 17:54:48.887500 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.887482    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 17:54:48.926545 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.925539    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.926545 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.925577    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.926545 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.925650    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.926545 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.925746    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.926545 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.925833    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33fac31c-bf67-40e2-95e0-d80bde4a4514-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.926545 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.925864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.926545 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.925888    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.926545 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.925913    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.926545 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.925950    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.926545 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.925985    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw7qn\" (UniqueName: \"kubernetes.io/projected/33fac31c-bf67-40e2-95e0-d80bde4a4514-kube-api-access-xw7qn\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.926545 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.926025    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.926545 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.926054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.926545 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.926152    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33fac31c-bf67-40e2-95e0-d80bde4a4514-config-out\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.927253 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.926178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.927253 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.926303    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.927253 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.926351    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-web-config\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.927253 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.926401    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.927705 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.927613    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-config\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.928141 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.928008    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:48.928141 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:48.928062    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.028600 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.028565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.028785 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.028619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-config\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.029185 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.029162    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.029290 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.029212    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.029290 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.029250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.029290 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.029271    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.029434 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.029302    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.029434 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.029347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.029434 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.029377    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33fac31c-bf67-40e2-95e0-d80bde4a4514-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.029434 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.029408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.029434 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.029431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.029678 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.029453    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.029678 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.029480    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.029678 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.029507    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw7qn\" (UniqueName: \"kubernetes.io/projected/33fac31c-bf67-40e2-95e0-d80bde4a4514-kube-api-access-xw7qn\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.030042 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.030015    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.030289 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.030267    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.032302 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.030391    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.032302 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.030432    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.032302 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.030476    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33fac31c-bf67-40e2-95e0-d80bde4a4514-config-out\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.032302 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.030501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.032302 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.030571    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.032302 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.030600    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-web-config\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.032302 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.030658    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.032302 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.031119    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.032302 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.031407    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.032302 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.031608    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.032302 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.032016    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.033655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.032548    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.033655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.032852    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-config\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.033655 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.033129    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33fac31c-bf67-40e2-95e0-d80bde4a4514-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.036103 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.036059    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.037690 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.037648    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33fac31c-bf67-40e2-95e0-d80bde4a4514-config-out\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.037690 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.037665    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.037866 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.037696    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.040071 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.039333    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-web-config\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.040071 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.039335    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.040071 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.039777    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.040071 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.039927    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.040764 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.040730    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.042430 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.042410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw7qn\" (UniqueName: \"kubernetes.io/projected/33fac31c-bf67-40e2-95e0-d80bde4a4514-kube-api-access-xw7qn\") pod \"prometheus-k8s-0\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.198138 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.198058    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:54:49.776250 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:49.776222    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pmx65"
Jul 12 17:54:50.408846 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:50.408720    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 17:54:50.412871 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:50.412838    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33fac31c_bf67_40e2_95e0_d80bde4a4514.slice/crio-237f5fcf233855ea47abd9aca619e9baafb0c7051d14602da647694215660e9d WatchSource:0}: Error finding container 237f5fcf233855ea47abd9aca619e9baafb0c7051d14602da647694215660e9d: Status 404 returned error can't find the container with id 237f5fcf233855ea47abd9aca619e9baafb0c7051d14602da647694215660e9d
Jul 12 17:54:50.853272 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:50.853188    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerStarted","Data":"c55723bfd3a21e026fd5f3810a02a35f06bebdc5af283bd1164ee35d45adeed3"}
Jul 12 17:54:50.853272 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:50.853268    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerStarted","Data":"c338468ef4cdb1f378e950b48fd2986b4f1050a9c35ae8de6846c1077fd96d39"}
Jul 12 17:54:50.853483 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:50.853312    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerStarted","Data":"eacd8db47e8a5dc2db72506e01d68fc14e364b7e78cfbd7178b28b58e2b3daf9"}
Jul 12 17:54:50.855559 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:50.855500    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm" event={"ID":"8bb373d7-9362-4040-a204-6fdad64d4d8a","Type":"ContainerStarted","Data":"4b2871634ab21848a7cad696f9b6a6decca52b8b07d42b29784004b974230059"}
Jul 12 17:54:50.855660 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:50.855605    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm" event={"ID":"8bb373d7-9362-4040-a204-6fdad64d4d8a","Type":"ContainerStarted","Data":"21595a8fc0c9145f79f42126524f96bbe572335666b3dbe83ea906d76649887f"}
Jul 12 17:54:50.855660 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:50.855650    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm" event={"ID":"8bb373d7-9362-4040-a204-6fdad64d4d8a","Type":"ContainerStarted","Data":"bb7dc697349071e90f682601480d9962096c985fdf078083b0c6658f67519a0f"}
Jul 12 17:54:50.857501 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:50.857452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerStarted","Data":"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"}
Jul 12 17:54:50.857620 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:50.857535    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerStarted","Data":"237f5fcf233855ea47abd9aca619e9baafb0c7051d14602da647694215660e9d"}
Jul 12 17:54:50.859132 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:50.859106    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v" event={"ID":"4140ff72-996c-40d9-b2f6-40e241f6894e","Type":"ContainerStarted","Data":"b0ffdb045c5b0b0bc403b5283ad28ccef6de37c07fb0b2fea8c4e4390a2da51f"}
Jul 12 17:54:50.925043 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:50.925000    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v" podStartSLOduration=2.504099072 podStartE2EDuration="4.924985739s" totalImagesPullingTime="2.420886667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:50.924559956 +0000 UTC m=+66.124679611" watchObservedRunningTime="2026-07-12 17:54:50.924985739 +0000 UTC m=+66.125105387"
Jul 12 17:54:51.050339 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.050306    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs\") pod \"network-metrics-daemon-k4llc\" (UID: \"4411fcad-827e-4fe9-9165-8eaba1e75226\") " pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:51.050481 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.050372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx29b\" (UniqueName: \"kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b\") pod \"network-check-target-ckkjw\" (UID: \"b4296c55-a6e8-4c34-93b6-3e72514d30bc\") " pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:51.052832 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.052806    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 12 17:54:51.052945 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.052839    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 12 17:54:51.063049 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.062980    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 12 17:54:51.063049 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.063042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4411fcad-827e-4fe9-9165-8eaba1e75226-metrics-certs\") pod \"network-metrics-daemon-k4llc\" (UID: \"4411fcad-827e-4fe9-9165-8eaba1e75226\") " pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:51.073357 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.073336    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx29b\" (UniqueName: \"kubernetes.io/projected/b4296c55-a6e8-4c34-93b6-3e72514d30bc-kube-api-access-vx29b\") pod \"network-check-target-ckkjw\" (UID: \"b4296c55-a6e8-4c34-93b6-3e72514d30bc\") " pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:51.135213 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.135146    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-84rf9\""
Jul 12 17:54:51.142338 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.142320    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h9zfm\""
Jul 12 17:54:51.142454 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.142372    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k4llc"
Jul 12 17:54:51.150482 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.150464    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:54:51.150853 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.150835    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:51.153327 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.153309    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 12 17:54:51.163349 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.163329    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d46ded0-0449-4d25-81bd-d3e7c5ae3a40-original-pull-secret\") pod \"global-pull-secret-syncer-97vpp\" (UID: \"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40\") " pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:51.311144 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.311120    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k4llc"]
Jul 12 17:54:51.330362 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.330335    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ckkjw"]
Jul 12 17:54:51.360328 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:51.360292    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4411fcad_827e_4fe9_9165_8eaba1e75226.slice/crio-991c81777e42ef1936f07b8b520f67ee3ef2508ba4b8960b0ef65fe5cca80125 WatchSource:0}: Error finding container 991c81777e42ef1936f07b8b520f67ee3ef2508ba4b8960b0ef65fe5cca80125: Status 404 returned error can't find the container with id 991c81777e42ef1936f07b8b520f67ee3ef2508ba4b8960b0ef65fe5cca80125
Jul 12 17:54:51.361294 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:51.361270    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4296c55_a6e8_4c34_93b6_3e72514d30bc.slice/crio-496dcb86c2896330e2aee6c592105908ee586dc9c95b1797f07f948d316f05ed WatchSource:0}: Error finding container 496dcb86c2896330e2aee6c592105908ee586dc9c95b1797f07f948d316f05ed: Status 404 returned error can't find the container with id 496dcb86c2896330e2aee6c592105908ee586dc9c95b1797f07f948d316f05ed
Jul 12 17:54:51.444963 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.444942    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-97vpp"
Jul 12 17:54:51.560277 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.560239    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:51.560388 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.560306    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:51.560787 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.560768    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f14ee62-aa17-47d0-8e04-eb49058e0e91-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:51.562479 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.562457    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f14ee62-aa17-47d0-8e04-eb49058e0e91-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xmkt\" (UID: \"3f14ee62-aa17-47d0-8e04-eb49058e0e91\") " pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:51.578676 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.578630    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-97vpp"]
Jul 12 17:54:51.582606 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:54:51.582586    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d46ded0_0449_4d25_81bd_d3e7c5ae3a40.slice/crio-cc480eb0097d33b35f1ae3d19f8c3f3928756b1d79884300c15a9fa4b95ef226 WatchSource:0}: Error finding container cc480eb0097d33b35f1ae3d19f8c3f3928756b1d79884300c15a9fa4b95ef226: Status 404 returned error can't find the container with id cc480eb0097d33b35f1ae3d19f8c3f3928756b1d79884300c15a9fa4b95ef226
Jul 12 17:54:51.607011 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.606987    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q8r2n\""
Jul 12 17:54:51.614964 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.614939    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8xmkt"
Jul 12 17:54:51.757780 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.757756    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8xmkt"]
Jul 12 17:54:51.869246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.869065    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerStarted","Data":"390c6afbc73943afe4da122cec5236846bb5f1b046b84706843e2b670e450ec2"}
Jul 12 17:54:51.869246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.869150    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerStarted","Data":"84ce9cd1ad9571482d5464db9fb4e93d0f21199f9e7865129e76c4b4702c8955"}
Jul 12 17:54:51.869246 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.869197    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerStarted","Data":"16c852e5df22e46844bff6ff840e1df67f9d2eaf852bc4985852936b37be063e"}
Jul 12 17:54:51.871028 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.870999    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq" event={"ID":"3b6d4c90-9b6b-4773-b866-88492d05373b","Type":"ContainerStarted","Data":"4d10689f56d733b0a4eadefb9a10c212783906d7977a6a1142135737f087839c"}
Jul 12 17:54:51.871419 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.871398    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"
Jul 12 17:54:51.872739 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.872662    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8xmkt" event={"ID":"3f14ee62-aa17-47d0-8e04-eb49058e0e91","Type":"ContainerStarted","Data":"f23ef9f124b56a8f3729e16a0ac4ddb9ffdb0c1f541f806d458e8dba9d991f0b"}
Jul 12 17:54:51.875645 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.875593    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-97vpp" event={"ID":"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40","Type":"ContainerStarted","Data":"cc480eb0097d33b35f1ae3d19f8c3f3928756b1d79884300c15a9fa4b95ef226"}
Jul 12 17:54:51.877233 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.877193    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ckkjw" event={"ID":"b4296c55-a6e8-4c34-93b6-3e72514d30bc","Type":"ContainerStarted","Data":"496dcb86c2896330e2aee6c592105908ee586dc9c95b1797f07f948d316f05ed"}
Jul 12 17:54:51.877450 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.877410    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq"
Jul 12 17:54:51.881006 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.880980    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm" event={"ID":"8bb373d7-9362-4040-a204-6fdad64d4d8a","Type":"ContainerStarted","Data":"a2ca268d5f7838fcf2660488f68fe474bee15ce129250580703338be85552b83"}
Jul 12 17:54:51.881087 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.881055    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm" event={"ID":"8bb373d7-9362-4040-a204-6fdad64d4d8a","Type":"ContainerStarted","Data":"dfd8d3d12d94ce73aec88238b9384da86b40fa26b12693eea4c7f450fec3669b"}
Jul 12 17:54:51.881144 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.881098    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm" event={"ID":"8bb373d7-9362-4040-a204-6fdad64d4d8a","Type":"ContainerStarted","Data":"76862bfd89409d3b025671466962d7ed34d9e2d7d905f1fcc75f544a62d5bd00"}
Jul 12 17:54:51.881189 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.881162    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:51.882800 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.882775    2578 generic.go:356] "Generic (PLEG): container finished" podID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerID="ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e" exitCode=0
Jul 12 17:54:51.883452 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.882950    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerDied","Data":"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"}
Jul 12 17:54:51.884310 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.884291    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k4llc" event={"ID":"4411fcad-827e-4fe9-9165-8eaba1e75226","Type":"ContainerStarted","Data":"991c81777e42ef1936f07b8b520f67ee3ef2508ba4b8960b0ef65fe5cca80125"}
Jul 12 17:54:51.900120 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.899142    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.456895729 podStartE2EDuration="8.899123659s" totalImagesPullingTime="5.421756201s" totalInitContainerRuntime="20.471729ms" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:51.897457245 +0000 UTC m=+67.097576940" watchObservedRunningTime="2026-07-12 17:54:51.899123659 +0000 UTC m=+67.099243314"
Jul 12 17:54:51.933772 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.933665    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm" podStartSLOduration=2.857186412 podStartE2EDuration="6.933650213s" totalImagesPullingTime="4.076463801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:51.930816728 +0000 UTC m=+67.130936411" watchObservedRunningTime="2026-07-12 17:54:51.933650213 +0000 UTC m=+67.133769867"
Jul 12 17:54:51.956098 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:51.956037    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-tp7vq" podStartSLOduration=2.576949538 podStartE2EDuration="4.956018913s" totalImagesPullingTime="2.379069375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:51.95457127 +0000 UTC m=+67.154690924" watchObservedRunningTime="2026-07-12 17:54:51.956018913 +0000 UTC m=+67.156138569"
Jul 12 17:54:52.803630 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:52.803558    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:54:52.894926 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:52.894889    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k4llc" event={"ID":"4411fcad-827e-4fe9-9165-8eaba1e75226","Type":"ContainerStarted","Data":"664aabc5da6a7bb3217def13b926ca6c055b6afd32698abf1bd97e1f2cdd95c5"}
Jul 12 17:54:52.895101 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:52.894967    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k4llc" event={"ID":"4411fcad-827e-4fe9-9165-8eaba1e75226","Type":"ContainerStarted","Data":"511dd7f52b28e416d8ca914953f4397f8516def9064808922c6c1938e169cb16"}
Jul 12 17:54:52.915771 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:52.915718    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k4llc" podStartSLOduration=65.86041388 podStartE2EDuration="1m6.915697527s" totalImagesPullingTime="1.055283647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:52.914269768 +0000 UTC m=+68.114389423" watchObservedRunningTime="2026-07-12 17:54:52.915697527 +0000 UTC m=+68.115817182"
Jul 12 17:54:57.845258 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:57.845218    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:57.845793 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:57.845283    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:57.850553 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:57.850530    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:54:57.901620 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:57.901592    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-b5cd8fc4c-6gtdm"
Jul 12 17:54:57.924690 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:54:57.924664    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:55:00.934128 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.934096    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8xmkt" event={"ID":"3f14ee62-aa17-47d0-8e04-eb49058e0e91","Type":"ContainerStarted","Data":"12a98b4643d3ebeefdc54ee4aeb311fb2c02c9ebea1b7c5ec7349f83566ad1bf"}
Jul 12 17:55:00.934448 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.934180    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8xmkt" event={"ID":"3f14ee62-aa17-47d0-8e04-eb49058e0e91","Type":"ContainerStarted","Data":"4dc1956791d00ff290405f59fcf0be4270d349a4e6c798c01a5783eff3106627"}
Jul 12 17:55:00.935435 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.935413    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-97vpp" event={"ID":"9d46ded0-0449-4d25-81bd-d3e7c5ae3a40","Type":"ContainerStarted","Data":"fcede6891f1919d409a3ed91a438f39b05bdffe590bb5a437139403888d6bc4a"}
Jul 12 17:55:00.936771 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.936751    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ckkjw" event={"ID":"b4296c55-a6e8-4c34-93b6-3e72514d30bc","Type":"ContainerStarted","Data":"6603f6f6db4ec37b4c378b4eec3f89e4f26c99fe2d6ea5ac5596903451c2ced4"}
Jul 12 17:55:00.936867 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.936853    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:55:00.940023 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.940005    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerStarted","Data":"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"}
Jul 12 17:55:00.940116 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.940065    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerStarted","Data":"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"}
Jul 12 17:55:00.940116 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.940113    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerStarted","Data":"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"}
Jul 12 17:55:00.940223 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.940175    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerStarted","Data":"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"}
Jul 12 17:55:00.940223 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.940218    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerStarted","Data":"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"}
Jul 12 17:55:00.940313 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.940258    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerStarted","Data":"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"}
Jul 12 17:55:00.952415 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.952378    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-97vpp" podStartSLOduration=66.468583862 podStartE2EDuration="1m14.952366495s" totalImagesPullingTime="8.483782633s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:55:00.950661329 +0000 UTC m=+76.150780997" watchObservedRunningTime="2026-07-12 17:55:00.952366495 +0000 UTC m=+76.152486183"
Jul 12 17:55:00.968401 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.968363    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ckkjw" podStartSLOduration=66.263676796 podStartE2EDuration="1m14.968350028s" totalImagesPullingTime="8.704673232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:55:00.967089168 +0000 UTC m=+76.167208822" watchObservedRunningTime="2026-07-12 17:55:00.968350028 +0000 UTC m=+76.168469681"
Jul 12 17:55:00.995826 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:00.995786    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.774076637 podStartE2EDuration="12.995773505s" totalImagesPullingTime="8.184219258s" totalInitContainerRuntime="37.47761ms" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:55:00.993581574 +0000 UTC m=+76.193701246" watchObservedRunningTime="2026-07-12 17:55:00.995773505 +0000 UTC m=+76.195893158"
Jul 12 17:55:03.956530 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:03.956485    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8xmkt" event={"ID":"3f14ee62-aa17-47d0-8e04-eb49058e0e91","Type":"ContainerStarted","Data":"3f2bc38c86adb1463ea1fd14785dd9659910c2433ad7780d4fe3a77caa22ef23"}
Jul 12 17:55:03.974053 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:03.974007    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8xmkt" podStartSLOduration=17.661632768 podStartE2EDuration="28.973993652s" totalImagesPullingTime="11.312360884s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:54:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:55:03.972691388 +0000 UTC m=+79.172811061" watchObservedRunningTime="2026-07-12 17:55:03.973993652 +0000 UTC m=+79.174113305"
Jul 12 17:55:04.199097 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:04.199064    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:55:07.342212 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:07.342102    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:55:07.342212 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:07.342152    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:55:13.873940 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:13.873875    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5dfb658bb9-htl8r" podUID="47f09a89-9355-4349-b2bb-b2ee714b737b" containerName="console" containerID="cri-o://29b25b098f8be389bc00ece6ed1fa270f694e47264c07d77ec67d273f803aa52" gracePeriod=15
Jul 12 17:55:13.997198 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:13.997169    2578 generic.go:356] "Generic (PLEG): container finished" podID="47f09a89-9355-4349-b2bb-b2ee714b737b" containerID="29b25b098f8be389bc00ece6ed1fa270f694e47264c07d77ec67d273f803aa52" exitCode=0
Jul 12 17:55:13.997293 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:13.997239    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfb658bb9-htl8r" event={"ID":"47f09a89-9355-4349-b2bb-b2ee714b737b","Type":"ContainerDied","Data":"29b25b098f8be389bc00ece6ed1fa270f694e47264c07d77ec67d273f803aa52"}
Jul 12 17:55:14.111107 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.111085    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:55:14.160041 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.159985    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-trusted-ca-bundle\") pod \"47f09a89-9355-4349-b2bb-b2ee714b737b\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") "
Jul 12 17:55:14.160041 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.160016    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-service-ca\") pod \"47f09a89-9355-4349-b2bb-b2ee714b737b\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") "
Jul 12 17:55:14.160192 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.160041    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-oauth-serving-cert\") pod \"47f09a89-9355-4349-b2bb-b2ee714b737b\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") "
Jul 12 17:55:14.160192 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.160072    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2zvw\" (UniqueName: \"kubernetes.io/projected/47f09a89-9355-4349-b2bb-b2ee714b737b-kube-api-access-x2zvw\") pod \"47f09a89-9355-4349-b2bb-b2ee714b737b\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") "
Jul 12 17:55:14.160192 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.160128    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47f09a89-9355-4349-b2bb-b2ee714b737b-console-oauth-config\") pod \"47f09a89-9355-4349-b2bb-b2ee714b737b\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") "
Jul 12 17:55:14.160192 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.160151    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47f09a89-9355-4349-b2bb-b2ee714b737b-console-serving-cert\") pod \"47f09a89-9355-4349-b2bb-b2ee714b737b\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") "
Jul 12 17:55:14.160353 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.160222    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-console-config\") pod \"47f09a89-9355-4349-b2bb-b2ee714b737b\" (UID: \"47f09a89-9355-4349-b2bb-b2ee714b737b\") "
Jul 12 17:55:14.160414 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.160369    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "47f09a89-9355-4349-b2bb-b2ee714b737b" (UID: "47f09a89-9355-4349-b2bb-b2ee714b737b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:55:14.160464 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.160411    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "47f09a89-9355-4349-b2bb-b2ee714b737b" (UID: "47f09a89-9355-4349-b2bb-b2ee714b737b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:55:14.160464 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.160423    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-service-ca" (OuterVolumeSpecName: "service-ca") pod "47f09a89-9355-4349-b2bb-b2ee714b737b" (UID: "47f09a89-9355-4349-b2bb-b2ee714b737b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:55:14.160577 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.160553    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-trusted-ca-bundle\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:55:14.160637 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.160621    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-service-ca\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:55:14.160681 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.160643    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-oauth-serving-cert\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:55:14.160817 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.160793    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-console-config" (OuterVolumeSpecName: "console-config") pod "47f09a89-9355-4349-b2bb-b2ee714b737b" (UID: "47f09a89-9355-4349-b2bb-b2ee714b737b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:55:14.162505 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.162484    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47f09a89-9355-4349-b2bb-b2ee714b737b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "47f09a89-9355-4349-b2bb-b2ee714b737b" (UID: "47f09a89-9355-4349-b2bb-b2ee714b737b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:55:14.162582 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.162493    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47f09a89-9355-4349-b2bb-b2ee714b737b-kube-api-access-x2zvw" (OuterVolumeSpecName: "kube-api-access-x2zvw") pod "47f09a89-9355-4349-b2bb-b2ee714b737b" (UID: "47f09a89-9355-4349-b2bb-b2ee714b737b"). InnerVolumeSpecName "kube-api-access-x2zvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 17:55:14.162582 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.162559    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47f09a89-9355-4349-b2bb-b2ee714b737b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "47f09a89-9355-4349-b2bb-b2ee714b737b" (UID: "47f09a89-9355-4349-b2bb-b2ee714b737b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:55:14.261198 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.261171    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/47f09a89-9355-4349-b2bb-b2ee714b737b-console-oauth-config\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:55:14.261198 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.261194    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/47f09a89-9355-4349-b2bb-b2ee714b737b-console-serving-cert\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:55:14.261329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.261205    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/47f09a89-9355-4349-b2bb-b2ee714b737b-console-config\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:55:14.261329 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:14.261213    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2zvw\" (UniqueName: \"kubernetes.io/projected/47f09a89-9355-4349-b2bb-b2ee714b737b-kube-api-access-x2zvw\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:55:15.002323 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:15.002293    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfb658bb9-htl8r" event={"ID":"47f09a89-9355-4349-b2bb-b2ee714b737b","Type":"ContainerDied","Data":"43a51e638dae9870c296979496de83c919ce4346226b3ddec6f3b1ecfbb1a0f0"}
Jul 12 17:55:15.002323 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:15.002314    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfb658bb9-htl8r"
Jul 12 17:55:15.002756 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:15.002352    2578 scope.go:117] "RemoveContainer" containerID="29b25b098f8be389bc00ece6ed1fa270f694e47264c07d77ec67d273f803aa52"
Jul 12 17:55:15.021010 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:15.020987    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dfb658bb9-htl8r"]
Jul 12 17:55:15.024141 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:15.024123    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5dfb658bb9-htl8r"]
Jul 12 17:55:15.324393 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:15.324332    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47f09a89-9355-4349-b2bb-b2ee714b737b" path="/var/lib/kubelet/pods/47f09a89-9355-4349-b2bb-b2ee714b737b/volumes"
Jul 12 17:55:27.347508 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:27.347483    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:55:27.351315 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:27.351291    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6b65dd6d44-rcs4v"
Jul 12 17:55:31.948171 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:31.948134    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ckkjw"
Jul 12 17:55:49.198843 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:49.198808    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:55:49.218000 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:49.217980    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:55:50.151041 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:55:50.151017    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:02.889587 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:02.889553    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 17:56:02.890106 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:02.889950    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="alertmanager" containerID="cri-o://eacd8db47e8a5dc2db72506e01d68fc14e364b7e78cfbd7178b28b58e2b3daf9" gracePeriod=120
Jul 12 17:56:02.890106 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:02.889991    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="kube-rbac-proxy-metric" containerID="cri-o://84ce9cd1ad9571482d5464db9fb4e93d0f21199f9e7865129e76c4b4702c8955" gracePeriod=120
Jul 12 17:56:02.890106 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:02.890009    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="kube-rbac-proxy-web" containerID="cri-o://c55723bfd3a21e026fd5f3810a02a35f06bebdc5af283bd1164ee35d45adeed3" gracePeriod=120
Jul 12 17:56:02.890106 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:02.890026    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="kube-rbac-proxy" containerID="cri-o://16c852e5df22e46844bff6ff840e1df67f9d2eaf852bc4985852936b37be063e" gracePeriod=120
Jul 12 17:56:02.890106 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:02.890054    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="config-reloader" containerID="cri-o://c338468ef4cdb1f378e950b48fd2986b4f1050a9c35ae8de6846c1077fd96d39" gracePeriod=120
Jul 12 17:56:02.890106 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:02.890067    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="prom-label-proxy" containerID="cri-o://390c6afbc73943afe4da122cec5236846bb5f1b046b84706843e2b670e450ec2" gracePeriod=120
Jul 12 17:56:03.189653 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:03.189624    2578 generic.go:356] "Generic (PLEG): container finished" podID="6140b6d8-80cb-4a67-834f-4a348801f202" containerID="390c6afbc73943afe4da122cec5236846bb5f1b046b84706843e2b670e450ec2" exitCode=0
Jul 12 17:56:03.189653 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:03.189648    2578 generic.go:356] "Generic (PLEG): container finished" podID="6140b6d8-80cb-4a67-834f-4a348801f202" containerID="84ce9cd1ad9571482d5464db9fb4e93d0f21199f9e7865129e76c4b4702c8955" exitCode=0
Jul 12 17:56:03.189653 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:03.189653    2578 generic.go:356] "Generic (PLEG): container finished" podID="6140b6d8-80cb-4a67-834f-4a348801f202" containerID="16c852e5df22e46844bff6ff840e1df67f9d2eaf852bc4985852936b37be063e" exitCode=0
Jul 12 17:56:03.189653 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:03.189659    2578 generic.go:356] "Generic (PLEG): container finished" podID="6140b6d8-80cb-4a67-834f-4a348801f202" containerID="c338468ef4cdb1f378e950b48fd2986b4f1050a9c35ae8de6846c1077fd96d39" exitCode=0
Jul 12 17:56:03.189653 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:03.189664    2578 generic.go:356] "Generic (PLEG): container finished" podID="6140b6d8-80cb-4a67-834f-4a348801f202" containerID="eacd8db47e8a5dc2db72506e01d68fc14e364b7e78cfbd7178b28b58e2b3daf9" exitCode=0
Jul 12 17:56:03.189922 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:03.189701    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerDied","Data":"390c6afbc73943afe4da122cec5236846bb5f1b046b84706843e2b670e450ec2"}
Jul 12 17:56:03.189922 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:03.189762    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerDied","Data":"84ce9cd1ad9571482d5464db9fb4e93d0f21199f9e7865129e76c4b4702c8955"}
Jul 12 17:56:03.189922 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:03.189788    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerDied","Data":"16c852e5df22e46844bff6ff840e1df67f9d2eaf852bc4985852936b37be063e"}
Jul 12 17:56:03.189922 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:03.189813    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerDied","Data":"c338468ef4cdb1f378e950b48fd2986b4f1050a9c35ae8de6846c1077fd96d39"}
Jul 12 17:56:03.189922 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:03.189836    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerDied","Data":"eacd8db47e8a5dc2db72506e01d68fc14e364b7e78cfbd7178b28b58e2b3daf9"}
Jul 12 17:56:04.134106 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.134086    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.197345 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.197316    2578 generic.go:356] "Generic (PLEG): container finished" podID="6140b6d8-80cb-4a67-834f-4a348801f202" containerID="c55723bfd3a21e026fd5f3810a02a35f06bebdc5af283bd1164ee35d45adeed3" exitCode=0
Jul 12 17:56:04.197478 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.197370    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerDied","Data":"c55723bfd3a21e026fd5f3810a02a35f06bebdc5af283bd1164ee35d45adeed3"}
Jul 12 17:56:04.197478 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.197425    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.197478 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.197438    2578 scope.go:117] "RemoveContainer" containerID="390c6afbc73943afe4da122cec5236846bb5f1b046b84706843e2b670e450ec2"
Jul 12 17:56:04.197617 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.197426    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6140b6d8-80cb-4a67-834f-4a348801f202","Type":"ContainerDied","Data":"241893933a65c0f5cab2cdcb78b644b786af9d16d5130491dd2c29278ea8a3e6"}
Jul 12 17:56:04.205690 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.205671    2578 scope.go:117] "RemoveContainer" containerID="84ce9cd1ad9571482d5464db9fb4e93d0f21199f9e7865129e76c4b4702c8955"
Jul 12 17:56:04.212720 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.212689    2578 scope.go:117] "RemoveContainer" containerID="16c852e5df22e46844bff6ff840e1df67f9d2eaf852bc4985852936b37be063e"
Jul 12 17:56:04.219384 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.219369    2578 scope.go:117] "RemoveContainer" containerID="c55723bfd3a21e026fd5f3810a02a35f06bebdc5af283bd1164ee35d45adeed3"
Jul 12 17:56:04.225786 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.225770    2578 scope.go:117] "RemoveContainer" containerID="c338468ef4cdb1f378e950b48fd2986b4f1050a9c35ae8de6846c1077fd96d39"
Jul 12 17:56:04.231973 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.231957    2578 scope.go:117] "RemoveContainer" containerID="eacd8db47e8a5dc2db72506e01d68fc14e364b7e78cfbd7178b28b58e2b3daf9"
Jul 12 17:56:04.237026 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237006    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-main-tls\") pod \"6140b6d8-80cb-4a67-834f-4a348801f202\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") "
Jul 12 17:56:04.237137 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237052    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-web-config\") pod \"6140b6d8-80cb-4a67-834f-4a348801f202\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") "
Jul 12 17:56:04.237137 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237098    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy-web\") pod \"6140b6d8-80cb-4a67-834f-4a348801f202\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") "
Jul 12 17:56:04.237137 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237122    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy-metric\") pod \"6140b6d8-80cb-4a67-834f-4a348801f202\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") "
Jul 12 17:56:04.237289 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237145    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-main-db\") pod \"6140b6d8-80cb-4a67-834f-4a348801f202\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") "
Jul 12 17:56:04.237289 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237182    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzllh\" (UniqueName: \"kubernetes.io/projected/6140b6d8-80cb-4a67-834f-4a348801f202-kube-api-access-wzllh\") pod \"6140b6d8-80cb-4a67-834f-4a348801f202\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") "
Jul 12 17:56:04.237289 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237230    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6140b6d8-80cb-4a67-834f-4a348801f202-config-out\") pod \"6140b6d8-80cb-4a67-834f-4a348801f202\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") "
Jul 12 17:56:04.237289 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237269    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-cluster-tls-config\") pod \"6140b6d8-80cb-4a67-834f-4a348801f202\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") "
Jul 12 17:56:04.237487 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237296    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-config-volume\") pod \"6140b6d8-80cb-4a67-834f-4a348801f202\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") "
Jul 12 17:56:04.237487 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237341    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-metrics-client-ca\") pod \"6140b6d8-80cb-4a67-834f-4a348801f202\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") "
Jul 12 17:56:04.237487 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237372    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy\") pod \"6140b6d8-80cb-4a67-834f-4a348801f202\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") "
Jul 12 17:56:04.237487 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237402    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6140b6d8-80cb-4a67-834f-4a348801f202-tls-assets\") pod \"6140b6d8-80cb-4a67-834f-4a348801f202\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") "
Jul 12 17:56:04.237487 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237428    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-trusted-ca-bundle\") pod \"6140b6d8-80cb-4a67-834f-4a348801f202\" (UID: \"6140b6d8-80cb-4a67-834f-4a348801f202\") "
Jul 12 17:56:04.237757 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237707    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "6140b6d8-80cb-4a67-834f-4a348801f202" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 17:56:04.238056 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.237999    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-main-db\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:04.238139 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.238061    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "6140b6d8-80cb-4a67-834f-4a348801f202" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:56:04.238139 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.238095    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "6140b6d8-80cb-4a67-834f-4a348801f202" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:56:04.238949 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.238697    2578 scope.go:117] "RemoveContainer" containerID="1607fc4b4c86f6119f083d2bae32f8fe7999c599b59b92caea5f25f044ecb51a"
Jul 12 17:56:04.240788 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.240491    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "6140b6d8-80cb-4a67-834f-4a348801f202" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:04.241267 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.241187    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "6140b6d8-80cb-4a67-834f-4a348801f202" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:04.241808 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.241760    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6140b6d8-80cb-4a67-834f-4a348801f202-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6140b6d8-80cb-4a67-834f-4a348801f202" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 17:56:04.241896 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.241760    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "6140b6d8-80cb-4a67-834f-4a348801f202" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:04.241896 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.241859    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-config-volume" (OuterVolumeSpecName: "config-volume") pod "6140b6d8-80cb-4a67-834f-4a348801f202" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:04.242206 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.242160    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6140b6d8-80cb-4a67-834f-4a348801f202-kube-api-access-wzllh" (OuterVolumeSpecName: "kube-api-access-wzllh") pod "6140b6d8-80cb-4a67-834f-4a348801f202" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202"). InnerVolumeSpecName "kube-api-access-wzllh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 17:56:04.242314 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.242295    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6140b6d8-80cb-4a67-834f-4a348801f202-config-out" (OuterVolumeSpecName: "config-out") pod "6140b6d8-80cb-4a67-834f-4a348801f202" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 17:56:04.242394 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.242376    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "6140b6d8-80cb-4a67-834f-4a348801f202" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:04.245223 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.245104    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "6140b6d8-80cb-4a67-834f-4a348801f202" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:04.251728 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.251708    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-web-config" (OuterVolumeSpecName: "web-config") pod "6140b6d8-80cb-4a67-834f-4a348801f202" (UID: "6140b6d8-80cb-4a67-834f-4a348801f202"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:04.258195 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.258174    2578 scope.go:117] "RemoveContainer" containerID="390c6afbc73943afe4da122cec5236846bb5f1b046b84706843e2b670e450ec2"
Jul 12 17:56:04.258433 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:04.258414    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"390c6afbc73943afe4da122cec5236846bb5f1b046b84706843e2b670e450ec2\": container with ID starting with 390c6afbc73943afe4da122cec5236846bb5f1b046b84706843e2b670e450ec2 not found: ID does not exist" containerID="390c6afbc73943afe4da122cec5236846bb5f1b046b84706843e2b670e450ec2"
Jul 12 17:56:04.258480 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.258441    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390c6afbc73943afe4da122cec5236846bb5f1b046b84706843e2b670e450ec2"} err="failed to get container status \"390c6afbc73943afe4da122cec5236846bb5f1b046b84706843e2b670e450ec2\": rpc error: code = NotFound desc = could not find container \"390c6afbc73943afe4da122cec5236846bb5f1b046b84706843e2b670e450ec2\": container with ID starting with 390c6afbc73943afe4da122cec5236846bb5f1b046b84706843e2b670e450ec2 not found: ID does not exist"
Jul 12 17:56:04.258480 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.258473    2578 scope.go:117] "RemoveContainer" containerID="84ce9cd1ad9571482d5464db9fb4e93d0f21199f9e7865129e76c4b4702c8955"
Jul 12 17:56:04.258726 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:04.258708    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84ce9cd1ad9571482d5464db9fb4e93d0f21199f9e7865129e76c4b4702c8955\": container with ID starting with 84ce9cd1ad9571482d5464db9fb4e93d0f21199f9e7865129e76c4b4702c8955 not found: ID does not exist" containerID="84ce9cd1ad9571482d5464db9fb4e93d0f21199f9e7865129e76c4b4702c8955"
Jul 12 17:56:04.258767 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.258732    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84ce9cd1ad9571482d5464db9fb4e93d0f21199f9e7865129e76c4b4702c8955"} err="failed to get container status \"84ce9cd1ad9571482d5464db9fb4e93d0f21199f9e7865129e76c4b4702c8955\": rpc error: code = NotFound desc = could not find container \"84ce9cd1ad9571482d5464db9fb4e93d0f21199f9e7865129e76c4b4702c8955\": container with ID starting with 84ce9cd1ad9571482d5464db9fb4e93d0f21199f9e7865129e76c4b4702c8955 not found: ID does not exist"
Jul 12 17:56:04.258767 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.258748    2578 scope.go:117] "RemoveContainer" containerID="16c852e5df22e46844bff6ff840e1df67f9d2eaf852bc4985852936b37be063e"
Jul 12 17:56:04.258958 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:04.258943    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16c852e5df22e46844bff6ff840e1df67f9d2eaf852bc4985852936b37be063e\": container with ID starting with 16c852e5df22e46844bff6ff840e1df67f9d2eaf852bc4985852936b37be063e not found: ID does not exist" containerID="16c852e5df22e46844bff6ff840e1df67f9d2eaf852bc4985852936b37be063e"
Jul 12 17:56:04.258991 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.258964    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16c852e5df22e46844bff6ff840e1df67f9d2eaf852bc4985852936b37be063e"} err="failed to get container status \"16c852e5df22e46844bff6ff840e1df67f9d2eaf852bc4985852936b37be063e\": rpc error: code = NotFound desc = could not find container \"16c852e5df22e46844bff6ff840e1df67f9d2eaf852bc4985852936b37be063e\": container with ID starting with 16c852e5df22e46844bff6ff840e1df67f9d2eaf852bc4985852936b37be063e not found: ID does not exist"
Jul 12 17:56:04.258991 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.258982    2578 scope.go:117] "RemoveContainer" containerID="c55723bfd3a21e026fd5f3810a02a35f06bebdc5af283bd1164ee35d45adeed3"
Jul 12 17:56:04.259182 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:04.259167    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c55723bfd3a21e026fd5f3810a02a35f06bebdc5af283bd1164ee35d45adeed3\": container with ID starting with c55723bfd3a21e026fd5f3810a02a35f06bebdc5af283bd1164ee35d45adeed3 not found: ID does not exist" containerID="c55723bfd3a21e026fd5f3810a02a35f06bebdc5af283bd1164ee35d45adeed3"
Jul 12 17:56:04.259226 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.259186    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c55723bfd3a21e026fd5f3810a02a35f06bebdc5af283bd1164ee35d45adeed3"} err="failed to get container status \"c55723bfd3a21e026fd5f3810a02a35f06bebdc5af283bd1164ee35d45adeed3\": rpc error: code = NotFound desc = could not find container \"c55723bfd3a21e026fd5f3810a02a35f06bebdc5af283bd1164ee35d45adeed3\": container with ID starting with c55723bfd3a21e026fd5f3810a02a35f06bebdc5af283bd1164ee35d45adeed3 not found: ID does not exist"
Jul 12 17:56:04.259226 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.259198    2578 scope.go:117] "RemoveContainer" containerID="c338468ef4cdb1f378e950b48fd2986b4f1050a9c35ae8de6846c1077fd96d39"
Jul 12 17:56:04.259434 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:04.259417    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c338468ef4cdb1f378e950b48fd2986b4f1050a9c35ae8de6846c1077fd96d39\": container with ID starting with c338468ef4cdb1f378e950b48fd2986b4f1050a9c35ae8de6846c1077fd96d39 not found: ID does not exist" containerID="c338468ef4cdb1f378e950b48fd2986b4f1050a9c35ae8de6846c1077fd96d39"
Jul 12 17:56:04.259493 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.259441    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c338468ef4cdb1f378e950b48fd2986b4f1050a9c35ae8de6846c1077fd96d39"} err="failed to get container status \"c338468ef4cdb1f378e950b48fd2986b4f1050a9c35ae8de6846c1077fd96d39\": rpc error: code = NotFound desc = could not find container \"c338468ef4cdb1f378e950b48fd2986b4f1050a9c35ae8de6846c1077fd96d39\": container with ID starting with c338468ef4cdb1f378e950b48fd2986b4f1050a9c35ae8de6846c1077fd96d39 not found: ID does not exist"
Jul 12 17:56:04.259493 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.259461    2578 scope.go:117] "RemoveContainer" containerID="eacd8db47e8a5dc2db72506e01d68fc14e364b7e78cfbd7178b28b58e2b3daf9"
Jul 12 17:56:04.259690 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:04.259674    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eacd8db47e8a5dc2db72506e01d68fc14e364b7e78cfbd7178b28b58e2b3daf9\": container with ID starting with eacd8db47e8a5dc2db72506e01d68fc14e364b7e78cfbd7178b28b58e2b3daf9 not found: ID does not exist" containerID="eacd8db47e8a5dc2db72506e01d68fc14e364b7e78cfbd7178b28b58e2b3daf9"
Jul 12 17:56:04.259731 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.259695    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eacd8db47e8a5dc2db72506e01d68fc14e364b7e78cfbd7178b28b58e2b3daf9"} err="failed to get container status \"eacd8db47e8a5dc2db72506e01d68fc14e364b7e78cfbd7178b28b58e2b3daf9\": rpc error: code = NotFound desc = could not find container \"eacd8db47e8a5dc2db72506e01d68fc14e364b7e78cfbd7178b28b58e2b3daf9\": container with ID starting with eacd8db47e8a5dc2db72506e01d68fc14e364b7e78cfbd7178b28b58e2b3daf9 not found: ID does not exist"
Jul 12 17:56:04.259731 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.259709    2578 scope.go:117] "RemoveContainer" containerID="1607fc4b4c86f6119f083d2bae32f8fe7999c599b59b92caea5f25f044ecb51a"
Jul 12 17:56:04.259898 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:04.259883    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1607fc4b4c86f6119f083d2bae32f8fe7999c599b59b92caea5f25f044ecb51a\": container with ID starting with 1607fc4b4c86f6119f083d2bae32f8fe7999c599b59b92caea5f25f044ecb51a not found: ID does not exist" containerID="1607fc4b4c86f6119f083d2bae32f8fe7999c599b59b92caea5f25f044ecb51a"
Jul 12 17:56:04.259934 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.259901    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1607fc4b4c86f6119f083d2bae32f8fe7999c599b59b92caea5f25f044ecb51a"} err="failed to get container status \"1607fc4b4c86f6119f083d2bae32f8fe7999c599b59b92caea5f25f044ecb51a\": rpc error: code = NotFound desc = could not find container \"1607fc4b4c86f6119f083d2bae32f8fe7999c599b59b92caea5f25f044ecb51a\": container with ID starting with 1607fc4b4c86f6119f083d2bae32f8fe7999c599b59b92caea5f25f044ecb51a not found: ID does not exist"
Jul 12 17:56:04.338486 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.338462    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-main-tls\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:04.338594 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.338505    2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-web-config\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:04.338594 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.338533    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:04.338594 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.338548    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:04.338594 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.338562    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzllh\" (UniqueName: \"kubernetes.io/projected/6140b6d8-80cb-4a67-834f-4a348801f202-kube-api-access-wzllh\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:04.338594 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.338574    2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6140b6d8-80cb-4a67-834f-4a348801f202-config-out\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:04.338594 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.338588    2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-cluster-tls-config\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:04.338801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.338600    2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-config-volume\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:04.338801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.338613    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-metrics-client-ca\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:04.338801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.338625    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6140b6d8-80cb-4a67-834f-4a348801f202-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:04.338801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.338637    2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6140b6d8-80cb-4a67-834f-4a348801f202-tls-assets\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:04.338801 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.338651    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6140b6d8-80cb-4a67-834f-4a348801f202-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:04.520972 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.520944    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 17:56:04.524987 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.524963    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 17:56:04.548579 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548560    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 17:56:04.548828 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548817    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47f09a89-9355-4349-b2bb-b2ee714b737b" containerName="console"
Jul 12 17:56:04.548871 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548830    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="47f09a89-9355-4349-b2bb-b2ee714b737b" containerName="console"
Jul 12 17:56:04.548871 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548862    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="init-config-reloader"
Jul 12 17:56:04.548871 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548867    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="init-config-reloader"
Jul 12 17:56:04.548954 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548875    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="kube-rbac-proxy-web"
Jul 12 17:56:04.548954 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548881    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="kube-rbac-proxy-web"
Jul 12 17:56:04.548954 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548886    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="prom-label-proxy"
Jul 12 17:56:04.548954 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548891    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="prom-label-proxy"
Jul 12 17:56:04.548954 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548899    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="kube-rbac-proxy-metric"
Jul 12 17:56:04.548954 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548904    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="kube-rbac-proxy-metric"
Jul 12 17:56:04.548954 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548914    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="alertmanager"
Jul 12 17:56:04.548954 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548918    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="alertmanager"
Jul 12 17:56:04.548954 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548924    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="kube-rbac-proxy"
Jul 12 17:56:04.548954 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548929    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="kube-rbac-proxy"
Jul 12 17:56:04.548954 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548940    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="config-reloader"
Jul 12 17:56:04.548954 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548945    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="config-reloader"
Jul 12 17:56:04.549318 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548988    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="alertmanager"
Jul 12 17:56:04.549318 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.548995    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="kube-rbac-proxy"
Jul 12 17:56:04.549318 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.549002    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="47f09a89-9355-4349-b2bb-b2ee714b737b" containerName="console"
Jul 12 17:56:04.549318 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.549007    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="config-reloader"
Jul 12 17:56:04.549318 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.549012    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="kube-rbac-proxy-metric"
Jul 12 17:56:04.549318 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.549017    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="prom-label-proxy"
Jul 12 17:56:04.549318 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.549022    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" containerName="kube-rbac-proxy-web"
Jul 12 17:56:04.554340 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.554320    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.557178 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.557152    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 12 17:56:04.557280 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.557153    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 12 17:56:04.557280 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.557221    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 12 17:56:04.557280 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.557259    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 12 17:56:04.557280 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.557272    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 12 17:56:04.557456 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.557265    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 12 17:56:04.557456 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.557346    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 12 17:56:04.557621 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.557606    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 12 17:56:04.557678 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.557664    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t22md\""
Jul 12 17:56:04.566780 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.562641    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 12 17:56:04.570683 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.570665    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 17:56:04.641474 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.641441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-web-config\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.641474 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.641478    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.641698 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.641496    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eab1750-c342-4045-a543-06e6cc42e3e4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.641698 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.641568    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5hvb\" (UniqueName: \"kubernetes.io/projected/7eab1750-c342-4045-a543-06e6cc42e3e4-kube-api-access-b5hvb\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.641698 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.641618    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7eab1750-c342-4045-a543-06e6cc42e3e4-config-out\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.641698 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.641642    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7eab1750-c342-4045-a543-06e6cc42e3e4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.641698 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.641673    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.641698 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.641693    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.641884 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.641711    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7eab1750-c342-4045-a543-06e6cc42e3e4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.641884 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.641762    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-config-volume\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.641884 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.641788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.641884 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.641817    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7eab1750-c342-4045-a543-06e6cc42e3e4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.641884 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.641845    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.742987 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.742958    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7eab1750-c342-4045-a543-06e6cc42e3e4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743142 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.742996    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743142 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.743019    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-web-config\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743142 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.743130    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743310 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.743168    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eab1750-c342-4045-a543-06e6cc42e3e4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743310 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.743194    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5hvb\" (UniqueName: \"kubernetes.io/projected/7eab1750-c342-4045-a543-06e6cc42e3e4-kube-api-access-b5hvb\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743310 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.743228    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7eab1750-c342-4045-a543-06e6cc42e3e4-config-out\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743310 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.743256    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7eab1750-c342-4045-a543-06e6cc42e3e4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743310 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.743280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743565 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.743310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743565 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.743342    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7eab1750-c342-4045-a543-06e6cc42e3e4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743565 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.743384    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-config-volume\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743565 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.743423    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743964 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.743909    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7eab1750-c342-4045-a543-06e6cc42e3e4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.743964 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.743954    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eab1750-c342-4045-a543-06e6cc42e3e4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.744146 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.744038    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7eab1750-c342-4045-a543-06e6cc42e3e4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.745870 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.745841    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7eab1750-c342-4045-a543-06e6cc42e3e4-config-out\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.746065 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.746043    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7eab1750-c342-4045-a543-06e6cc42e3e4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.746310 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.746286    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.746487 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.746467    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-config-volume\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.746600 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.746581    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.746639 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.746598    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-web-config\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.746736 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.746720    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.746833 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.746811    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.747765 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.747749    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7eab1750-c342-4045-a543-06e6cc42e3e4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.750484 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.750464    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5hvb\" (UniqueName: \"kubernetes.io/projected/7eab1750-c342-4045-a543-06e6cc42e3e4-kube-api-access-b5hvb\") pod \"alertmanager-main-0\" (UID: \"7eab1750-c342-4045-a543-06e6cc42e3e4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.868559 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.868489    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 12 17:56:04.995343 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:04.995306    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 12 17:56:04.998991 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:56:04.998967    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eab1750_c342_4045_a543_06e6cc42e3e4.slice/crio-0ab1198521bfc08afe95430aff18f92b9d9dd22d0dab21b5e4db061ccf2d6528 WatchSource:0}: Error finding container 0ab1198521bfc08afe95430aff18f92b9d9dd22d0dab21b5e4db061ccf2d6528: Status 404 returned error can't find the container with id 0ab1198521bfc08afe95430aff18f92b9d9dd22d0dab21b5e4db061ccf2d6528
Jul 12 17:56:05.202735 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:05.202710    2578 generic.go:356] "Generic (PLEG): container finished" podID="7eab1750-c342-4045-a543-06e6cc42e3e4" containerID="c3e5b9050d65a7f865eb7093802e8528ecd72ffa1ba8e4a827a2b32f82f47d6a" exitCode=0
Jul 12 17:56:05.203069 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:05.202805    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7eab1750-c342-4045-a543-06e6cc42e3e4","Type":"ContainerDied","Data":"c3e5b9050d65a7f865eb7093802e8528ecd72ffa1ba8e4a827a2b32f82f47d6a"}
Jul 12 17:56:05.203069 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:05.202881    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7eab1750-c342-4045-a543-06e6cc42e3e4","Type":"ContainerStarted","Data":"0ab1198521bfc08afe95430aff18f92b9d9dd22d0dab21b5e4db061ccf2d6528"}
Jul 12 17:56:05.324858 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:05.324835    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6140b6d8-80cb-4a67-834f-4a348801f202" path="/var/lib/kubelet/pods/6140b6d8-80cb-4a67-834f-4a348801f202/volumes"
Jul 12 17:56:06.211479 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.211446    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7eab1750-c342-4045-a543-06e6cc42e3e4","Type":"ContainerStarted","Data":"98a8cd4105470377a377698519d2cedd168f270d8622eb9f21bfe268d338f622"}
Jul 12 17:56:06.211881 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.211556    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7eab1750-c342-4045-a543-06e6cc42e3e4","Type":"ContainerStarted","Data":"b3a2125a56829de4cfedd5d7f712d9e97b9cc6f1fe76461927cfa49f04c385ae"}
Jul 12 17:56:06.211881 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.211589    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7eab1750-c342-4045-a543-06e6cc42e3e4","Type":"ContainerStarted","Data":"a534fa57876c16c54fa743ab56cc1b52c5f9812ff74f3df55de25182aa55221b"}
Jul 12 17:56:06.211881 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.211620    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7eab1750-c342-4045-a543-06e6cc42e3e4","Type":"ContainerStarted","Data":"5265cd0c50fba8421c956ac0861f317228180263bc52c92763716c7c46dcb6e8"}
Jul 12 17:56:06.211881 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.211645    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7eab1750-c342-4045-a543-06e6cc42e3e4","Type":"ContainerStarted","Data":"4223fb36379f437a4ab8d9864a6f73a624602326fff545db5d73b6597363b96c"}
Jul 12 17:56:06.211881 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.211679    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7eab1750-c342-4045-a543-06e6cc42e3e4","Type":"ContainerStarted","Data":"fa0c73121b1b170eefea4e17a21d3a77b4c5c5e8c4b62d95d8d34b58c45e30ef"}
Jul 12 17:56:06.237944 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.237834    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.219404219 podStartE2EDuration="2.237815811s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.411592ms" isStatefulPod=false podCreationTimestamp="2026-07-12 17:56:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:56:06.236834722 +0000 UTC m=+141.436954374" watchObservedRunningTime="2026-07-12 17:56:06.237815811 +0000 UTC m=+141.437935466"
Jul 12 17:56:06.924983 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.924943    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-787fd96d66-mlrfx"]
Jul 12 17:56:06.927371 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.927351    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:06.930431 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.930194    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 12 17:56:06.930431 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.930228    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 12 17:56:06.930431 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.930261    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 12 17:56:06.930431 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.930285    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 12 17:56:06.930431 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.930229    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 12 17:56:06.930431 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.930235    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-b42qc\""
Jul 12 17:56:06.935617 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.935215    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 12 17:56:06.941719 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.941699    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-787fd96d66-mlrfx"]
Jul 12 17:56:06.960259 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.960237    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/609cdcb8-a619-420b-9ed8-38c2d34a9e99-serving-certs-ca-bundle\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:06.960365 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.960263    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzt87\" (UniqueName: \"kubernetes.io/projected/609cdcb8-a619-420b-9ed8-38c2d34a9e99-kube-api-access-dzt87\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:06.960365 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.960291    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/609cdcb8-a619-420b-9ed8-38c2d34a9e99-telemeter-trusted-ca-bundle\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:06.960365 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.960311    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/609cdcb8-a619-420b-9ed8-38c2d34a9e99-telemeter-client-tls\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:06.960546 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.960393    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/609cdcb8-a619-420b-9ed8-38c2d34a9e99-federate-client-tls\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:06.960546 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.960436    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/609cdcb8-a619-420b-9ed8-38c2d34a9e99-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:06.960546 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.960467    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/609cdcb8-a619-420b-9ed8-38c2d34a9e99-metrics-client-ca\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:06.960654 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:06.960550    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/609cdcb8-a619-420b-9ed8-38c2d34a9e99-secret-telemeter-client\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.061333 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.061288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/609cdcb8-a619-420b-9ed8-38c2d34a9e99-serving-certs-ca-bundle\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.061333 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.061324    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzt87\" (UniqueName: \"kubernetes.io/projected/609cdcb8-a619-420b-9ed8-38c2d34a9e99-kube-api-access-dzt87\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.061579 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.061347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/609cdcb8-a619-420b-9ed8-38c2d34a9e99-telemeter-trusted-ca-bundle\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.061579 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.061375    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/609cdcb8-a619-420b-9ed8-38c2d34a9e99-telemeter-client-tls\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.061579 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.061424    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/609cdcb8-a619-420b-9ed8-38c2d34a9e99-federate-client-tls\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.061579 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.061471    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/609cdcb8-a619-420b-9ed8-38c2d34a9e99-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.061579 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.061509    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/609cdcb8-a619-420b-9ed8-38c2d34a9e99-metrics-client-ca\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.061579 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.061562    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/609cdcb8-a619-420b-9ed8-38c2d34a9e99-secret-telemeter-client\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.062159 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.062132    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/609cdcb8-a619-420b-9ed8-38c2d34a9e99-serving-certs-ca-bundle\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.062300 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.062249    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/609cdcb8-a619-420b-9ed8-38c2d34a9e99-telemeter-trusted-ca-bundle\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.062300 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.062266    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/609cdcb8-a619-420b-9ed8-38c2d34a9e99-metrics-client-ca\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.064602 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.064339    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/609cdcb8-a619-420b-9ed8-38c2d34a9e99-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.064602 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.064424    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/609cdcb8-a619-420b-9ed8-38c2d34a9e99-secret-telemeter-client\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.064602 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.064511    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/609cdcb8-a619-420b-9ed8-38c2d34a9e99-federate-client-tls\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.065439 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.065419    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/609cdcb8-a619-420b-9ed8-38c2d34a9e99-telemeter-client-tls\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.069736 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.069710    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzt87\" (UniqueName: \"kubernetes.io/projected/609cdcb8-a619-420b-9ed8-38c2d34a9e99-kube-api-access-dzt87\") pod \"telemeter-client-787fd96d66-mlrfx\" (UID: \"609cdcb8-a619-420b-9ed8-38c2d34a9e99\") " pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.185488 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.185454    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 17:56:07.185859 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.185826    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="prometheus" containerID="cri-o://c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75" gracePeriod=600
Jul 12 17:56:07.185974 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.185850    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="thanos-sidecar" containerID="cri-o://0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70" gracePeriod=600
Jul 12 17:56:07.185974 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.185870    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="kube-rbac-proxy-web" containerID="cri-o://3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062" gracePeriod=600
Jul 12 17:56:07.185974 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.185850    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="kube-rbac-proxy" containerID="cri-o://22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738" gracePeriod=600
Jul 12 17:56:07.185974 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.185952    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="kube-rbac-proxy-thanos" containerID="cri-o://97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb" gracePeriod=600
Jul 12 17:56:07.186201 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.186028    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="config-reloader" containerID="cri-o://ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f" gracePeriod=600
Jul 12 17:56:07.238187 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.238164    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx"
Jul 12 17:56:07.410792 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.410759    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-787fd96d66-mlrfx"]
Jul 12 17:56:07.411947 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:56:07.411914    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod609cdcb8_a619_420b_9ed8_38c2d34a9e99.slice/crio-beedd9c39ddd2d17e901f65551d71639a5c94ce46e65846f817b802584b68e2b WatchSource:0}: Error finding container beedd9c39ddd2d17e901f65551d71639a5c94ce46e65846f817b802584b68e2b: Status 404 returned error can't find the container with id beedd9c39ddd2d17e901f65551d71639a5c94ce46e65846f817b802584b68e2b
Jul 12 17:56:07.478724 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.478697    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:07.567395 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567359    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567395 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567395    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-kube-rbac-proxy\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567619 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567416    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-2\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567619 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567444    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-kubelet-serving-ca-bundle\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567619 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567487    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-db\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567619 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567547    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-trusted-ca-bundle\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567619 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567574    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-grpc-tls\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567619 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567606    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-config\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567909 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567639    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-0\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567909 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567666    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-1\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567909 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567704    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567909 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567729    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-web-config\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567909 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567756    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw7qn\" (UniqueName: \"kubernetes.io/projected/33fac31c-bf67-40e2-95e0-d80bde4a4514-kube-api-access-xw7qn\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567909 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567784    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-tls\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567909 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567858    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-metrics-client-certs\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567909 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567885    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-metrics-client-ca\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.567909 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567910    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-thanos-prometheus-http-client-file\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.568313 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567934    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33fac31c-bf67-40e2-95e0-d80bde4a4514-config-out\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.568313 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567956    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33fac31c-bf67-40e2-95e0-d80bde4a4514-tls-assets\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.568313 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.568000    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-serving-certs-ca-bundle\") pod \"33fac31c-bf67-40e2-95e0-d80bde4a4514\" (UID: \"33fac31c-bf67-40e2-95e0-d80bde4a4514\") "
Jul 12 17:56:07.568871 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567748    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:56:07.568971 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567872    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:56:07.568971 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.567996    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:56:07.568971 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.568489    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:56:07.569382 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.569357    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:56:07.569702 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.569663    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 17:56:07.569868 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.569840    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:56:07.570165 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.570140    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:56:07.570724 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.570676    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-config" (OuterVolumeSpecName: "config") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:07.571528 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.571429    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:07.571634 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.571610    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:07.572053 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.572020    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:07.572141 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.572123    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33fac31c-bf67-40e2-95e0-d80bde4a4514-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 17:56:07.572287 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.572252    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:07.572392 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.572321    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:07.572457 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.572401    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33fac31c-bf67-40e2-95e0-d80bde4a4514-config-out" (OuterVolumeSpecName: "config-out") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 17:56:07.572935 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.572913    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33fac31c-bf67-40e2-95e0-d80bde4a4514-kube-api-access-xw7qn" (OuterVolumeSpecName: "kube-api-access-xw7qn") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "kube-api-access-xw7qn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 17:56:07.572994 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.572928    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:07.573115 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.573099    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:07.581615 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.581594    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-web-config" (OuterVolumeSpecName: "web-config") pod "33fac31c-bf67-40e2-95e0-d80bde4a4514" (UID: "33fac31c-bf67-40e2-95e0-d80bde4a4514"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:07.669367 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669325    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-db\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669367 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669364    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669367 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669374    2578 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-grpc-tls\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669367 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669382    2578 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-config\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669390    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669398    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669408    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669418    2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-web-config\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669425    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw7qn\" (UniqueName: \"kubernetes.io/projected/33fac31c-bf67-40e2-95e0-d80bde4a4514-kube-api-access-xw7qn\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669433    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669441    2578 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-metrics-client-certs\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669449    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-metrics-client-ca\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669458    2578 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669467    2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33fac31c-bf67-40e2-95e0-d80bde4a4514-config-out\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669474    2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33fac31c-bf67-40e2-95e0-d80bde4a4514-tls-assets\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669481    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669489    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669497    2578 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/33fac31c-bf67-40e2-95e0-d80bde4a4514-secret-kube-rbac-proxy\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669504    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:07.669601 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:07.669536    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fac31c-bf67-40e2-95e0-d80bde4a4514-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:08.223208 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.223174    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx" event={"ID":"609cdcb8-a619-420b-9ed8-38c2d34a9e99","Type":"ContainerStarted","Data":"beedd9c39ddd2d17e901f65551d71639a5c94ce46e65846f817b802584b68e2b"}
Jul 12 17:56:08.226178 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226149    2578 generic.go:356] "Generic (PLEG): container finished" podID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerID="97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb" exitCode=0
Jul 12 17:56:08.226178 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226175    2578 generic.go:356] "Generic (PLEG): container finished" podID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerID="22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738" exitCode=0
Jul 12 17:56:08.226178 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226183    2578 generic.go:356] "Generic (PLEG): container finished" podID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerID="3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062" exitCode=0
Jul 12 17:56:08.226410 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226189    2578 generic.go:356] "Generic (PLEG): container finished" podID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerID="0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70" exitCode=0
Jul 12 17:56:08.226410 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226195    2578 generic.go:356] "Generic (PLEG): container finished" podID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerID="ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f" exitCode=0
Jul 12 17:56:08.226410 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226200    2578 generic.go:356] "Generic (PLEG): container finished" podID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerID="c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75" exitCode=0
Jul 12 17:56:08.226410 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226227    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.226410 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226243    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerDied","Data":"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"}
Jul 12 17:56:08.226410 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226328    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerDied","Data":"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"}
Jul 12 17:56:08.226410 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226373    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerDied","Data":"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"}
Jul 12 17:56:08.226410 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226393    2578 scope.go:117] "RemoveContainer" containerID="97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"
Jul 12 17:56:08.226410 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226414    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerDied","Data":"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"}
Jul 12 17:56:08.226834 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226453    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerDied","Data":"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"}
Jul 12 17:56:08.226834 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226494    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerDied","Data":"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"}
Jul 12 17:56:08.226834 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.226557    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"33fac31c-bf67-40e2-95e0-d80bde4a4514","Type":"ContainerDied","Data":"237f5fcf233855ea47abd9aca619e9baafb0c7051d14602da647694215660e9d"}
Jul 12 17:56:08.233999 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.233984    2578 scope.go:117] "RemoveContainer" containerID="22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"
Jul 12 17:56:08.240609 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.240590    2578 scope.go:117] "RemoveContainer" containerID="3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"
Jul 12 17:56:08.246969 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.246950    2578 scope.go:117] "RemoveContainer" containerID="0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"
Jul 12 17:56:08.249170 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.249151    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 17:56:08.253818 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.253798    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 17:56:08.256699 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.256673    2578 scope.go:117] "RemoveContainer" containerID="ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"
Jul 12 17:56:08.265237 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.265212    2578 scope.go:117] "RemoveContainer" containerID="c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"
Jul 12 17:56:08.274661 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.274640    2578 scope.go:117] "RemoveContainer" containerID="ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"
Jul 12 17:56:08.278840 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.278805    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 17:56:08.279230 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279209    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="kube-rbac-proxy"
Jul 12 17:56:08.279230 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279232    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="kube-rbac-proxy"
Jul 12 17:56:08.279377 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279247    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="init-config-reloader"
Jul 12 17:56:08.279377 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279257    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="init-config-reloader"
Jul 12 17:56:08.279377 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279274    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="kube-rbac-proxy-web"
Jul 12 17:56:08.279377 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279283    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="kube-rbac-proxy-web"
Jul 12 17:56:08.279377 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279292    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="kube-rbac-proxy-thanos"
Jul 12 17:56:08.279377 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279301    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="kube-rbac-proxy-thanos"
Jul 12 17:56:08.279377 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279315    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="config-reloader"
Jul 12 17:56:08.279377 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279325    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="config-reloader"
Jul 12 17:56:08.279377 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279346    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="prometheus"
Jul 12 17:56:08.279377 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279355    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="prometheus"
Jul 12 17:56:08.279377 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279367    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="thanos-sidecar"
Jul 12 17:56:08.279377 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279376    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="thanos-sidecar"
Jul 12 17:56:08.280013 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279455    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="kube-rbac-proxy"
Jul 12 17:56:08.280013 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279470    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="kube-rbac-proxy-thanos"
Jul 12 17:56:08.280013 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279479    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="config-reloader"
Jul 12 17:56:08.280013 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279489    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="thanos-sidecar"
Jul 12 17:56:08.280013 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279497    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="kube-rbac-proxy-web"
Jul 12 17:56:08.280013 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.279507    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" containerName="prometheus"
Jul 12 17:56:08.283719 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.283703    2578 scope.go:117] "RemoveContainer" containerID="97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"
Jul 12 17:56:08.284070 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:08.284020    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb\": container with ID starting with 97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb not found: ID does not exist" containerID="97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"
Jul 12 17:56:08.284153 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.284075    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"} err="failed to get container status \"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb\": rpc error: code = NotFound desc = could not find container \"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb\": container with ID starting with 97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb not found: ID does not exist"
Jul 12 17:56:08.284153 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.284093    2578 scope.go:117] "RemoveContainer" containerID="22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"
Jul 12 17:56:08.284367 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:08.284346    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738\": container with ID starting with 22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738 not found: ID does not exist" containerID="22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"
Jul 12 17:56:08.284453 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.284373    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"} err="failed to get container status \"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738\": rpc error: code = NotFound desc = could not find container \"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738\": container with ID starting with 22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738 not found: ID does not exist"
Jul 12 17:56:08.284453 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.284395    2578 scope.go:117] "RemoveContainer" containerID="3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"
Jul 12 17:56:08.284900 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:08.284874    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062\": container with ID starting with 3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062 not found: ID does not exist" containerID="3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"
Jul 12 17:56:08.284963 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.284907    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"} err="failed to get container status \"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062\": rpc error: code = NotFound desc = could not find container \"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062\": container with ID starting with 3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062 not found: ID does not exist"
Jul 12 17:56:08.284963 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.284925    2578 scope.go:117] "RemoveContainer" containerID="0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"
Jul 12 17:56:08.285164 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.285146    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.285736 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:08.285271    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70\": container with ID starting with 0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70 not found: ID does not exist" containerID="0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"
Jul 12 17:56:08.285736 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.285303    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"} err="failed to get container status \"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70\": rpc error: code = NotFound desc = could not find container \"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70\": container with ID starting with 0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70 not found: ID does not exist"
Jul 12 17:56:08.285736 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.285324    2578 scope.go:117] "RemoveContainer" containerID="ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"
Jul 12 17:56:08.285951 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:08.285748    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f\": container with ID starting with ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f not found: ID does not exist" containerID="ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"
Jul 12 17:56:08.285951 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.285799    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"} err="failed to get container status \"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f\": rpc error: code = NotFound desc = could not find container \"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f\": container with ID starting with ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f not found: ID does not exist"
Jul 12 17:56:08.285951 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.285821    2578 scope.go:117] "RemoveContainer" containerID="c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"
Jul 12 17:56:08.286197 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:08.286167    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75\": container with ID starting with c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75 not found: ID does not exist" containerID="c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"
Jul 12 17:56:08.286253 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.286206    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"} err="failed to get container status \"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75\": rpc error: code = NotFound desc = could not find container \"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75\": container with ID starting with c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75 not found: ID does not exist"
Jul 12 17:56:08.286253 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.286226    2578 scope.go:117] "RemoveContainer" containerID="ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"
Jul 12 17:56:08.286557 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:08.286498    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e\": container with ID starting with ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e not found: ID does not exist" containerID="ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"
Jul 12 17:56:08.286669 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.286645    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"} err="failed to get container status \"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e\": rpc error: code = NotFound desc = could not find container \"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e\": container with ID starting with ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e not found: ID does not exist"
Jul 12 17:56:08.286716 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.286671    2578 scope.go:117] "RemoveContainer" containerID="97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"
Jul 12 17:56:08.286927 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.286896    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"} err="failed to get container status \"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb\": rpc error: code = NotFound desc = could not find container \"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb\": container with ID starting with 97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb not found: ID does not exist"
Jul 12 17:56:08.286927 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.286920    2578 scope.go:117] "RemoveContainer" containerID="22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"
Jul 12 17:56:08.287258 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.287228    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"} err="failed to get container status \"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738\": rpc error: code = NotFound desc = could not find container \"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738\": container with ID starting with 22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738 not found: ID does not exist"
Jul 12 17:56:08.287333 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.287279    2578 scope.go:117] "RemoveContainer" containerID="3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"
Jul 12 17:56:08.287572 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.287545    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"} err="failed to get container status \"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062\": rpc error: code = NotFound desc = could not find container \"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062\": container with ID starting with 3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062 not found: ID does not exist"
Jul 12 17:56:08.287674 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.287574    2578 scope.go:117] "RemoveContainer" containerID="0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"
Jul 12 17:56:08.287914 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.287829    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 12 17:56:08.287914 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.287831    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6bfscmt615doe\""
Jul 12 17:56:08.288118 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.287988    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"} err="failed to get container status \"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70\": rpc error: code = NotFound desc = could not find container \"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70\": container with ID starting with 0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70 not found: ID does not exist"
Jul 12 17:56:08.288118 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288010    2578 scope.go:117] "RemoveContainer" containerID="ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"
Jul 12 17:56:08.288574 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288266    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"} err="failed to get container status \"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f\": rpc error: code = NotFound desc = could not find container \"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f\": container with ID starting with ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f not found: ID does not exist"
Jul 12 17:56:08.288574 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288289    2578 scope.go:117] "RemoveContainer" containerID="c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"
Jul 12 17:56:08.288574 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288322    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 12 17:56:08.288574 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288355    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 12 17:56:08.288574 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288398    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 12 17:56:08.288574 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288410    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 12 17:56:08.288574 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288421    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bpsr8\""
Jul 12 17:56:08.288574 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288438    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 12 17:56:08.288574 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288362    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 12 17:56:08.288574 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288381    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 12 17:56:08.288574 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288359    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 12 17:56:08.288574 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288267    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 12 17:56:08.289208 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288637    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"} err="failed to get container status \"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75\": rpc error: code = NotFound desc = could not find container \"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75\": container with ID starting with c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75 not found: ID does not exist"
Jul 12 17:56:08.289208 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288658    2578 scope.go:117] "RemoveContainer" containerID="ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"
Jul 12 17:56:08.289208 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288900    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"} err="failed to get container status \"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e\": rpc error: code = NotFound desc = could not find container \"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e\": container with ID starting with ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e not found: ID does not exist"
Jul 12 17:56:08.289208 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288922    2578 scope.go:117] "RemoveContainer" containerID="97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"
Jul 12 17:56:08.289208 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.288984    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 12 17:56:08.289208 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.289141    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 12 17:56:08.289208 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.289182    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"} err="failed to get container status \"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb\": rpc error: code = NotFound desc = could not find container \"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb\": container with ID starting with 97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb not found: ID does not exist"
Jul 12 17:56:08.289208 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.289207    2578 scope.go:117] "RemoveContainer" containerID="22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"
Jul 12 17:56:08.289620 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.289574    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"} err="failed to get container status \"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738\": rpc error: code = NotFound desc = could not find container \"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738\": container with ID starting with 22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738 not found: ID does not exist"
Jul 12 17:56:08.289620 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.289599    2578 scope.go:117] "RemoveContainer" containerID="3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"
Jul 12 17:56:08.290001 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.289981    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"} err="failed to get container status \"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062\": rpc error: code = NotFound desc = could not find container \"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062\": container with ID starting with 3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062 not found: ID does not exist"
Jul 12 17:56:08.290079 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.290001    2578 scope.go:117] "RemoveContainer" containerID="0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"
Jul 12 17:56:08.290341 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.290316    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"} err="failed to get container status \"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70\": rpc error: code = NotFound desc = could not find container \"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70\": container with ID starting with 0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70 not found: ID does not exist"
Jul 12 17:56:08.290413 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.290344    2578 scope.go:117] "RemoveContainer" containerID="ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"
Jul 12 17:56:08.290728 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.290703    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"} err="failed to get container status \"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f\": rpc error: code = NotFound desc = could not find container \"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f\": container with ID starting with ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f not found: ID does not exist"
Jul 12 17:56:08.290810 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.290732    2578 scope.go:117] "RemoveContainer" containerID="c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"
Jul 12 17:56:08.291029 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.291004    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"} err="failed to get container status \"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75\": rpc error: code = NotFound desc = could not find container \"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75\": container with ID starting with c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75 not found: ID does not exist"
Jul 12 17:56:08.291104 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.291030    2578 scope.go:117] "RemoveContainer" containerID="ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"
Jul 12 17:56:08.291309 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.291293    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 12 17:56:08.291826 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.291801    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"} err="failed to get container status \"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e\": rpc error: code = NotFound desc = could not find container \"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e\": container with ID starting with ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e not found: ID does not exist"
Jul 12 17:56:08.292006 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.291827    2578 scope.go:117] "RemoveContainer" containerID="97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"
Jul 12 17:56:08.292159 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.292136    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"} err="failed to get container status \"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb\": rpc error: code = NotFound desc = could not find container \"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb\": container with ID starting with 97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb not found: ID does not exist"
Jul 12 17:56:08.292236 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.292162    2578 scope.go:117] "RemoveContainer" containerID="22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"
Jul 12 17:56:08.292791 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.292765    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"} err="failed to get container status \"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738\": rpc error: code = NotFound desc = could not find container \"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738\": container with ID starting with 22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738 not found: ID does not exist"
Jul 12 17:56:08.292889 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.292791    2578 scope.go:117] "RemoveContainer" containerID="3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"
Jul 12 17:56:08.293255 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.293226    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"} err="failed to get container status \"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062\": rpc error: code = NotFound desc = could not find container \"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062\": container with ID starting with 3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062 not found: ID does not exist"
Jul 12 17:56:08.293255 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.293255    2578 scope.go:117] "RemoveContainer" containerID="0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"
Jul 12 17:56:08.294136 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.294113    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 12 17:56:08.294272 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.294147    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"} err="failed to get container status \"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70\": rpc error: code = NotFound desc = could not find container \"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70\": container with ID starting with 0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70 not found: ID does not exist"
Jul 12 17:56:08.294272 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.294186    2578 scope.go:117] "RemoveContainer" containerID="ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"
Jul 12 17:56:08.296114 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.295209    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"} err="failed to get container status \"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f\": rpc error: code = NotFound desc = could not find container \"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f\": container with ID starting with ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f not found: ID does not exist"
Jul 12 17:56:08.296114 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.295237    2578 scope.go:117] "RemoveContainer" containerID="c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"
Jul 12 17:56:08.296114 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.295679    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"} err="failed to get container status \"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75\": rpc error: code = NotFound desc = could not find container \"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75\": container with ID starting with c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75 not found: ID does not exist"
Jul 12 17:56:08.296114 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.295722    2578 scope.go:117] "RemoveContainer" containerID="ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"
Jul 12 17:56:08.296312 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.296254    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"} err="failed to get container status \"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e\": rpc error: code = NotFound desc = could not find container \"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e\": container with ID starting with ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e not found: ID does not exist"
Jul 12 17:56:08.296312 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.296278    2578 scope.go:117] "RemoveContainer" containerID="97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"
Jul 12 17:56:08.296713 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.296690    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"} err="failed to get container status \"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb\": rpc error: code = NotFound desc = could not find container \"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb\": container with ID starting with 97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb not found: ID does not exist"
Jul 12 17:56:08.296822 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.296715    2578 scope.go:117] "RemoveContainer" containerID="22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"
Jul 12 17:56:08.296999 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.296973    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"} err="failed to get container status \"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738\": rpc error: code = NotFound desc = could not find container \"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738\": container with ID starting with 22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738 not found: ID does not exist"
Jul 12 17:56:08.297088 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.297001    2578 scope.go:117] "RemoveContainer" containerID="3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"
Jul 12 17:56:08.297253 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.297229    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 17:56:08.297345 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.297260    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"} err="failed to get container status \"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062\": rpc error: code = NotFound desc = could not find container \"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062\": container with ID starting with 3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062 not found: ID does not exist"
Jul 12 17:56:08.297345 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.297283    2578 scope.go:117] "RemoveContainer" containerID="0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"
Jul 12 17:56:08.297488 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.297469    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"} err="failed to get container status \"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70\": rpc error: code = NotFound desc = could not find container \"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70\": container with ID starting with 0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70 not found: ID does not exist"
Jul 12 17:56:08.297570 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.297489    2578 scope.go:117] "RemoveContainer" containerID="ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"
Jul 12 17:56:08.297747 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.297723    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"} err="failed to get container status \"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f\": rpc error: code = NotFound desc = could not find container \"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f\": container with ID starting with ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f not found: ID does not exist"
Jul 12 17:56:08.297817 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.297748    2578 scope.go:117] "RemoveContainer" containerID="c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"
Jul 12 17:56:08.298034 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.298005    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"} err="failed to get container status \"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75\": rpc error: code = NotFound desc = could not find container \"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75\": container with ID starting with c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75 not found: ID does not exist"
Jul 12 17:56:08.298155 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.298037    2578 scope.go:117] "RemoveContainer" containerID="ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"
Jul 12 17:56:08.298450 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.298414    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"} err="failed to get container status \"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e\": rpc error: code = NotFound desc = could not find container \"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e\": container with ID starting with ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e not found: ID does not exist"
Jul 12 17:56:08.298450 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.298440    2578 scope.go:117] "RemoveContainer" containerID="97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"
Jul 12 17:56:08.299137 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.299047    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb"} err="failed to get container status \"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb\": rpc error: code = NotFound desc = could not find container \"97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb\": container with ID starting with 97a3a46dade8238e5fa7164a93b7c44e7854edea94b40109a013756099d30bdb not found: ID does not exist"
Jul 12 17:56:08.299137 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.299073    2578 scope.go:117] "RemoveContainer" containerID="22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"
Jul 12 17:56:08.299676 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.299417    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738"} err="failed to get container status \"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738\": rpc error: code = NotFound desc = could not find container \"22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738\": container with ID starting with 22d72c5dc9f6f841f9f0c6ccf1948e105844be9ccdd1646919bbbe8871805738 not found: ID does not exist"
Jul 12 17:56:08.299676 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.299442    2578 scope.go:117] "RemoveContainer" containerID="3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"
Jul 12 17:56:08.299819 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.299723    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062"} err="failed to get container status \"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062\": rpc error: code = NotFound desc = could not find container \"3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062\": container with ID starting with 3fa6abf14c9235a3a97cbd2d5f495bd9bbddab5c5bbab89826cb710f174b5062 not found: ID does not exist"
Jul 12 17:56:08.299819 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.299742    2578 scope.go:117] "RemoveContainer" containerID="0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"
Jul 12 17:56:08.300000 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.299979    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70"} err="failed to get container status \"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70\": rpc error: code = NotFound desc = could not find container \"0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70\": container with ID starting with 0ceb4976e13bb8fa08b59f3aeae6da9b68dfb0671491a12ef5e8b862d628fb70 not found: ID does not exist"
Jul 12 17:56:08.300062 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.300001    2578 scope.go:117] "RemoveContainer" containerID="ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"
Jul 12 17:56:08.300255 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.300232    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f"} err="failed to get container status \"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f\": rpc error: code = NotFound desc = could not find container \"ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f\": container with ID starting with ac8a5b6c2b30222206815dd5c474400c0155cf7f14369e8b203b26a45558b60f not found: ID does not exist"
Jul 12 17:56:08.300255 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.300255    2578 scope.go:117] "RemoveContainer" containerID="c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"
Jul 12 17:56:08.300497 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.300470    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75"} err="failed to get container status \"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75\": rpc error: code = NotFound desc = could not find container \"c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75\": container with ID starting with c1231f8a3a6832752365872a98ac901464dcde50ff87c964f89ffa950a3fbf75 not found: ID does not exist"
Jul 12 17:56:08.300602 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.300497    2578 scope.go:117] "RemoveContainer" containerID="ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"
Jul 12 17:56:08.300832 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.300811    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e"} err="failed to get container status \"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e\": rpc error: code = NotFound desc = could not find container \"ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e\": container with ID starting with ea671d62cb1386e0ed3d5cd8b4b62476ed2370016cf5aecb1e285c28d181ea5e not found: ID does not exist"
Jul 12 17:56:08.375790 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.375721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e6bc8957-a8df-47e6-8467-520bb945a6e2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.375790 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.375770    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.375951 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.375811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-web-config\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.375951 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.375837    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.375951 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.375864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.375951 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.375909    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376105 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.375980    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376105 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.376049    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376105 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.376077    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376105 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.376098    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376242 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.376146    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376242 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.376178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376242 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.376208    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376338 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.376238    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376338 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.376281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e6bc8957-a8df-47e6-8467-520bb945a6e2-config-out\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376338 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.376323    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376467 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.376363    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nt2k\" (UniqueName: \"kubernetes.io/projected/e6bc8957-a8df-47e6-8467-520bb945a6e2-kube-api-access-6nt2k\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376467 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.376392    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376467 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.376431    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.376467 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.376460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-config\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.477715 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.477683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e6bc8957-a8df-47e6-8467-520bb945a6e2-config-out\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.477863 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.477739    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.477863 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.477768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nt2k\" (UniqueName: \"kubernetes.io/projected/e6bc8957-a8df-47e6-8467-520bb945a6e2-kube-api-access-6nt2k\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.477863 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.477791    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.477863 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.477819    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.477863 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.477849    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-config\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478121 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.477874    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e6bc8957-a8df-47e6-8467-520bb945a6e2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478121 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.477897    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478121 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.477935    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-web-config\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478121 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.477964    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478121 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.477987    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478121 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.478041    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478121 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.478070    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478121 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.478097    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478479 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.478127    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478479 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.478153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478479 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.478197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478479 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.478225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478479 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.478249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478479 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.478275    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.478479 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.478427    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.480864 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.479093    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.480864 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.479432    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.480864 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.480057    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.480864 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.480405    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.480864 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.480557    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.480864 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.480590    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e6bc8957-a8df-47e6-8467-520bb945a6e2-config-out\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.481609 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.481531    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.484645 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.484364    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e6bc8957-a8df-47e6-8467-520bb945a6e2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.484843 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.484820    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.484915 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.484872    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.484915 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.484909    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.485251 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.485230    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-web-config\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.485326 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.485254    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.485388 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.485370    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e6bc8957-a8df-47e6-8467-520bb945a6e2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.485425 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.485379    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.485425 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.485374    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.485591 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.485552    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-config\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.485870 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.485849    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e6bc8957-a8df-47e6-8467-520bb945a6e2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.487746 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.487726    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nt2k\" (UniqueName: \"kubernetes.io/projected/e6bc8957-a8df-47e6-8467-520bb945a6e2-kube-api-access-6nt2k\") pod \"prometheus-k8s-0\" (UID: \"e6bc8957-a8df-47e6-8467-520bb945a6e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.596951 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.596925    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:08.736400 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:08.736374    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 12 17:56:09.056849 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:56:09.056818    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6bc8957_a8df_47e6_8467_520bb945a6e2.slice/crio-a27568fa5b82ea71024e9b96e177cf54131527fa648e6c9da659631da883da7f WatchSource:0}: Error finding container a27568fa5b82ea71024e9b96e177cf54131527fa648e6c9da659631da883da7f: Status 404 returned error can't find the container with id a27568fa5b82ea71024e9b96e177cf54131527fa648e6c9da659631da883da7f
Jul 12 17:56:09.232137 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:09.232093    2578 generic.go:356] "Generic (PLEG): container finished" podID="e6bc8957-a8df-47e6-8467-520bb945a6e2" containerID="e80a881fa713fdaa47a7653f7add1fcc369d51c1be536e880c274d99ec0f774d" exitCode=0
Jul 12 17:56:09.232244 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:09.232210    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6bc8957-a8df-47e6-8467-520bb945a6e2","Type":"ContainerDied","Data":"e80a881fa713fdaa47a7653f7add1fcc369d51c1be536e880c274d99ec0f774d"}
Jul 12 17:56:09.232306 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:09.232285    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6bc8957-a8df-47e6-8467-520bb945a6e2","Type":"ContainerStarted","Data":"a27568fa5b82ea71024e9b96e177cf54131527fa648e6c9da659631da883da7f"}
Jul 12 17:56:09.234063 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:09.234036    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx" event={"ID":"609cdcb8-a619-420b-9ed8-38c2d34a9e99","Type":"ContainerStarted","Data":"2a6f4f452fd8cf83a68fed2ab3123643b3b8638def1db75aa03ac6e8ffa6eef2"}
Jul 12 17:56:09.325750 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:09.325722    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33fac31c-bf67-40e2-95e0-d80bde4a4514" path="/var/lib/kubelet/pods/33fac31c-bf67-40e2-95e0-d80bde4a4514/volumes"
Jul 12 17:56:10.242212 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:10.242178    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6bc8957-a8df-47e6-8467-520bb945a6e2","Type":"ContainerStarted","Data":"c96006eca058834276fd4f5bb8e2f0302055a53289b1339ab68b230af7067c75"}
Jul 12 17:56:10.242373 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:10.242265    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6bc8957-a8df-47e6-8467-520bb945a6e2","Type":"ContainerStarted","Data":"60d232db394d0fcc963b630ad16e97c39ecb1a2d544be6f82d0b1729f4aac2af"}
Jul 12 17:56:10.242373 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:10.242309    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6bc8957-a8df-47e6-8467-520bb945a6e2","Type":"ContainerStarted","Data":"816ea430b0a811f097c09431f2bda9a7d6143a01687bcd0a2cb9a60835789f78"}
Jul 12 17:56:10.242373 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:10.242344    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6bc8957-a8df-47e6-8467-520bb945a6e2","Type":"ContainerStarted","Data":"852b410e1b49978c53345209cc2ef6fe5118feb37b215ff4740d31a73c5bb7f6"}
Jul 12 17:56:10.242536 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:10.242380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6bc8957-a8df-47e6-8467-520bb945a6e2","Type":"ContainerStarted","Data":"9d0637b7b937e17f40891e19ae22894618f42f69f3ff018e96bcee65daea7e1f"}
Jul 12 17:56:10.242536 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:10.242417    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e6bc8957-a8df-47e6-8467-520bb945a6e2","Type":"ContainerStarted","Data":"aca8ccde717aac933b1ce29a03d222d759f71a656187bb67348746ab3b5057de"}
Jul 12 17:56:10.244087 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:10.244056    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx" event={"ID":"609cdcb8-a619-420b-9ed8-38c2d34a9e99","Type":"ContainerStarted","Data":"ef9be1838e4ece3ca883193b055595f28987faff82fd30ceab3612f0e4999ca3"}
Jul 12 17:56:10.244173 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:10.244112    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx" event={"ID":"609cdcb8-a619-420b-9ed8-38c2d34a9e99","Type":"ContainerStarted","Data":"e7513e31c4e3a992b7d5a949fbf7024407a7d8a8a1c487e200deee0d0ed8daf9"}
Jul 12 17:56:10.271642 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:10.271598    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.248258754 podStartE2EDuration="2.271583619s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.324865ms" isStatefulPod=false podCreationTimestamp="2026-07-12 17:56:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:56:10.269441859 +0000 UTC m=+145.469561532" watchObservedRunningTime="2026-07-12 17:56:10.271583619 +0000 UTC m=+145.471703271"
Jul 12 17:56:10.289007 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:10.288964    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-787fd96d66-mlrfx" podStartSLOduration=2.6189510670000002 podStartE2EDuration="4.288949441s" totalImagesPullingTime="1.669998374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:56:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:56:10.287368752 +0000 UTC m=+145.487488455" watchObservedRunningTime="2026-07-12 17:56:10.288949441 +0000 UTC m=+145.489069110"
Jul 12 17:56:13.597291 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:13.597252    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:56:22.656176 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:22.656145    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ff4489496-rqmjp"]
Jul 12 17:56:47.676203 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:47.676164    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7ff4489496-rqmjp" podUID="87079739-c667-438f-8d31-9e674f1002d9" containerName="console" containerID="cri-o://4fd80aa17de73108e42e5d1cb71241dfc22dc8b551b6cf3ddf3ad382666e318b" gracePeriod=15
Jul 12 17:56:47.914320 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:47.914293    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:56:48.072624 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.072597    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-trusted-ca-bundle\") pod \"87079739-c667-438f-8d31-9e674f1002d9\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") "
Jul 12 17:56:48.072624 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.072625    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87079739-c667-438f-8d31-9e674f1002d9-console-oauth-config\") pod \"87079739-c667-438f-8d31-9e674f1002d9\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") "
Jul 12 17:56:48.072828 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.072641    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87079739-c667-438f-8d31-9e674f1002d9-console-serving-cert\") pod \"87079739-c667-438f-8d31-9e674f1002d9\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") "
Jul 12 17:56:48.072828 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.072666    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-console-config\") pod \"87079739-c667-438f-8d31-9e674f1002d9\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") "
Jul 12 17:56:48.072828 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.072689    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6876w\" (UniqueName: \"kubernetes.io/projected/87079739-c667-438f-8d31-9e674f1002d9-kube-api-access-6876w\") pod \"87079739-c667-438f-8d31-9e674f1002d9\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") "
Jul 12 17:56:48.072828 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.072800    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-service-ca\") pod \"87079739-c667-438f-8d31-9e674f1002d9\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") "
Jul 12 17:56:48.073036 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.072833    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-oauth-serving-cert\") pod \"87079739-c667-438f-8d31-9e674f1002d9\" (UID: \"87079739-c667-438f-8d31-9e674f1002d9\") "
Jul 12 17:56:48.073121 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.073096    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "87079739-c667-438f-8d31-9e674f1002d9" (UID: "87079739-c667-438f-8d31-9e674f1002d9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:56:48.073172 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.073117    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-console-config" (OuterVolumeSpecName: "console-config") pod "87079739-c667-438f-8d31-9e674f1002d9" (UID: "87079739-c667-438f-8d31-9e674f1002d9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:56:48.073287 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.073180    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-service-ca" (OuterVolumeSpecName: "service-ca") pod "87079739-c667-438f-8d31-9e674f1002d9" (UID: "87079739-c667-438f-8d31-9e674f1002d9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:56:48.073334 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.073316    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "87079739-c667-438f-8d31-9e674f1002d9" (UID: "87079739-c667-438f-8d31-9e674f1002d9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 17:56:48.074782 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.074759    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87079739-c667-438f-8d31-9e674f1002d9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "87079739-c667-438f-8d31-9e674f1002d9" (UID: "87079739-c667-438f-8d31-9e674f1002d9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:48.075117 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.075092    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87079739-c667-438f-8d31-9e674f1002d9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "87079739-c667-438f-8d31-9e674f1002d9" (UID: "87079739-c667-438f-8d31-9e674f1002d9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 17:56:48.075205 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.075144    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87079739-c667-438f-8d31-9e674f1002d9-kube-api-access-6876w" (OuterVolumeSpecName: "kube-api-access-6876w") pod "87079739-c667-438f-8d31-9e674f1002d9" (UID: "87079739-c667-438f-8d31-9e674f1002d9"). InnerVolumeSpecName "kube-api-access-6876w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 17:56:48.173666 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.173645    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-console-config\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:48.173666 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.173665    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6876w\" (UniqueName: \"kubernetes.io/projected/87079739-c667-438f-8d31-9e674f1002d9-kube-api-access-6876w\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:48.173779 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.173674    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-service-ca\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:48.173779 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.173682    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-oauth-serving-cert\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:48.173779 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.173690    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87079739-c667-438f-8d31-9e674f1002d9-trusted-ca-bundle\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:48.173779 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.173697    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87079739-c667-438f-8d31-9e674f1002d9-console-oauth-config\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:48.173779 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.173705    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87079739-c667-438f-8d31-9e674f1002d9-console-serving-cert\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 17:56:48.392839 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.392771    2578 generic.go:356] "Generic (PLEG): container finished" podID="87079739-c667-438f-8d31-9e674f1002d9" containerID="4fd80aa17de73108e42e5d1cb71241dfc22dc8b551b6cf3ddf3ad382666e318b" exitCode=0
Jul 12 17:56:48.392839 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.392826    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff4489496-rqmjp"
Jul 12 17:56:48.392977 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.392851    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff4489496-rqmjp" event={"ID":"87079739-c667-438f-8d31-9e674f1002d9","Type":"ContainerDied","Data":"4fd80aa17de73108e42e5d1cb71241dfc22dc8b551b6cf3ddf3ad382666e318b"}
Jul 12 17:56:48.392977 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.392898    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff4489496-rqmjp" event={"ID":"87079739-c667-438f-8d31-9e674f1002d9","Type":"ContainerDied","Data":"8637b3e3966a00b93f441f97b12dfb1e22d12fb3ab0901799b50bcddeb694906"}
Jul 12 17:56:48.392977 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.392917    2578 scope.go:117] "RemoveContainer" containerID="4fd80aa17de73108e42e5d1cb71241dfc22dc8b551b6cf3ddf3ad382666e318b"
Jul 12 17:56:48.401166 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.401149    2578 scope.go:117] "RemoveContainer" containerID="4fd80aa17de73108e42e5d1cb71241dfc22dc8b551b6cf3ddf3ad382666e318b"
Jul 12 17:56:48.401413 ip-10-0-131-122 kubenswrapper[2578]: E0712 17:56:48.401394    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fd80aa17de73108e42e5d1cb71241dfc22dc8b551b6cf3ddf3ad382666e318b\": container with ID starting with 4fd80aa17de73108e42e5d1cb71241dfc22dc8b551b6cf3ddf3ad382666e318b not found: ID does not exist" containerID="4fd80aa17de73108e42e5d1cb71241dfc22dc8b551b6cf3ddf3ad382666e318b"
Jul 12 17:56:48.401470 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.401420    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fd80aa17de73108e42e5d1cb71241dfc22dc8b551b6cf3ddf3ad382666e318b"} err="failed to get container status \"4fd80aa17de73108e42e5d1cb71241dfc22dc8b551b6cf3ddf3ad382666e318b\": rpc error: code = NotFound desc = could not find container \"4fd80aa17de73108e42e5d1cb71241dfc22dc8b551b6cf3ddf3ad382666e318b\": container with ID starting with 4fd80aa17de73108e42e5d1cb71241dfc22dc8b551b6cf3ddf3ad382666e318b not found: ID does not exist"
Jul 12 17:56:48.412630 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.412611    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ff4489496-rqmjp"]
Jul 12 17:56:48.416216 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:48.416194    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7ff4489496-rqmjp"]
Jul 12 17:56:49.324624 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:56:49.324588    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87079739-c667-438f-8d31-9e674f1002d9" path="/var/lib/kubelet/pods/87079739-c667-438f-8d31-9e674f1002d9/volumes"
Jul 12 17:57:08.597278 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:57:08.597246    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:57:08.612137 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:57:08.612118    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:57:09.487018 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:57:09.486993    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 12 17:58:45.162652 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:45.162628    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 17:58:45.166254 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:45.166231    2578 kubelet.go:1648] "Image garbage collection succeeded"
Jul 12 17:58:45.170347 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:45.170330    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 17:58:46.988854 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:46.988828    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g"]
Jul 12 17:58:46.990984 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:46.989166    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87079739-c667-438f-8d31-9e674f1002d9" containerName="console"
Jul 12 17:58:46.990984 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:46.989178    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="87079739-c667-438f-8d31-9e674f1002d9" containerName="console"
Jul 12 17:58:46.990984 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:46.989238    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="87079739-c667-438f-8d31-9e674f1002d9" containerName="console"
Jul 12 17:58:46.991928 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:46.991913    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g"
Jul 12 17:58:46.995598 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:46.995573    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 12 17:58:46.995737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:46.995605    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-nhlr8\""
Jul 12 17:58:46.995737 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:46.995607    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 12 17:58:47.000376 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.000353    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g"]
Jul 12 17:58:47.089084 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.089057    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/672bf4e7-8189-4f59-b53d-7e6f792772e6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8hn8g\" (UID: \"672bf4e7-8189-4f59-b53d-7e6f792772e6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g"
Jul 12 17:58:47.089229 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.089091    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6pgz\" (UniqueName: \"kubernetes.io/projected/672bf4e7-8189-4f59-b53d-7e6f792772e6-kube-api-access-n6pgz\") pod \"openshift-lws-operator-fd8ccff4c-8hn8g\" (UID: \"672bf4e7-8189-4f59-b53d-7e6f792772e6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g"
Jul 12 17:58:47.190181 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.190143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/672bf4e7-8189-4f59-b53d-7e6f792772e6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8hn8g\" (UID: \"672bf4e7-8189-4f59-b53d-7e6f792772e6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g"
Jul 12 17:58:47.190181 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.190197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6pgz\" (UniqueName: \"kubernetes.io/projected/672bf4e7-8189-4f59-b53d-7e6f792772e6-kube-api-access-n6pgz\") pod \"openshift-lws-operator-fd8ccff4c-8hn8g\" (UID: \"672bf4e7-8189-4f59-b53d-7e6f792772e6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g"
Jul 12 17:58:47.190581 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.190563    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/672bf4e7-8189-4f59-b53d-7e6f792772e6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8hn8g\" (UID: \"672bf4e7-8189-4f59-b53d-7e6f792772e6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g"
Jul 12 17:58:47.199852 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.199820    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6pgz\" (UniqueName: \"kubernetes.io/projected/672bf4e7-8189-4f59-b53d-7e6f792772e6-kube-api-access-n6pgz\") pod \"openshift-lws-operator-fd8ccff4c-8hn8g\" (UID: \"672bf4e7-8189-4f59-b53d-7e6f792772e6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g"
Jul 12 17:58:47.313028 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.312937    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g"
Jul 12 17:58:47.380117 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.378057    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z4zjw"]
Jul 12 17:58:47.383039 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.383015    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z4zjw"
Jul 12 17:58:47.387095 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.386814    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 12 17:58:47.387095 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.386999    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 12 17:58:47.387237 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.387131    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-x986x\""
Jul 12 17:58:47.391020 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.390625    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z4zjw"]
Jul 12 17:58:47.456758 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.456733    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g"]
Jul 12 17:58:47.458716 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:58:47.458691    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod672bf4e7_8189_4f59_b53d_7e6f792772e6.slice/crio-fc76e56abf365468cfa0701f022848d48df52204346bf947d89d5bd3b4f827b7 WatchSource:0}: Error finding container fc76e56abf365468cfa0701f022848d48df52204346bf947d89d5bd3b4f827b7: Status 404 returned error can't find the container with id fc76e56abf365468cfa0701f022848d48df52204346bf947d89d5bd3b4f827b7
Jul 12 17:58:47.492331 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.492302    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3950f4fb-1ab0-4d37-9954-c12238bcac2c-bound-sa-token\") pod \"cert-manager-55b4d954d5-z4zjw\" (UID: \"3950f4fb-1ab0-4d37-9954-c12238bcac2c\") " pod="cert-manager/cert-manager-55b4d954d5-z4zjw"
Jul 12 17:58:47.492471 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.492354    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsc64\" (UniqueName: \"kubernetes.io/projected/3950f4fb-1ab0-4d37-9954-c12238bcac2c-kube-api-access-rsc64\") pod \"cert-manager-55b4d954d5-z4zjw\" (UID: \"3950f4fb-1ab0-4d37-9954-c12238bcac2c\") " pod="cert-manager/cert-manager-55b4d954d5-z4zjw"
Jul 12 17:58:47.593732 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.593643    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3950f4fb-1ab0-4d37-9954-c12238bcac2c-bound-sa-token\") pod \"cert-manager-55b4d954d5-z4zjw\" (UID: \"3950f4fb-1ab0-4d37-9954-c12238bcac2c\") " pod="cert-manager/cert-manager-55b4d954d5-z4zjw"
Jul 12 17:58:47.593732 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.593729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsc64\" (UniqueName: \"kubernetes.io/projected/3950f4fb-1ab0-4d37-9954-c12238bcac2c-kube-api-access-rsc64\") pod \"cert-manager-55b4d954d5-z4zjw\" (UID: \"3950f4fb-1ab0-4d37-9954-c12238bcac2c\") " pod="cert-manager/cert-manager-55b4d954d5-z4zjw"
Jul 12 17:58:47.602002 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.601972    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3950f4fb-1ab0-4d37-9954-c12238bcac2c-bound-sa-token\") pod \"cert-manager-55b4d954d5-z4zjw\" (UID: \"3950f4fb-1ab0-4d37-9954-c12238bcac2c\") " pod="cert-manager/cert-manager-55b4d954d5-z4zjw"
Jul 12 17:58:47.602101 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.602005    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsc64\" (UniqueName: \"kubernetes.io/projected/3950f4fb-1ab0-4d37-9954-c12238bcac2c-kube-api-access-rsc64\") pod \"cert-manager-55b4d954d5-z4zjw\" (UID: \"3950f4fb-1ab0-4d37-9954-c12238bcac2c\") " pod="cert-manager/cert-manager-55b4d954d5-z4zjw"
Jul 12 17:58:47.698924 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.698902    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z4zjw"
Jul 12 17:58:47.819189 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.819028    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z4zjw"]
Jul 12 17:58:47.821584 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:58:47.821559    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3950f4fb_1ab0_4d37_9954_c12238bcac2c.slice/crio-64da4931044bb8969143de7934333168590cfd0e51dab797d23d4f75cae61e3a WatchSource:0}: Error finding container 64da4931044bb8969143de7934333168590cfd0e51dab797d23d4f75cae61e3a: Status 404 returned error can't find the container with id 64da4931044bb8969143de7934333168590cfd0e51dab797d23d4f75cae61e3a
Jul 12 17:58:47.823537 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.823499    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 17:58:47.855401 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.855338    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g" event={"ID":"672bf4e7-8189-4f59-b53d-7e6f792772e6","Type":"ContainerStarted","Data":"fc76e56abf365468cfa0701f022848d48df52204346bf947d89d5bd3b4f827b7"}
Jul 12 17:58:47.856428 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:47.856406    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z4zjw" event={"ID":"3950f4fb-1ab0-4d37-9954-c12238bcac2c","Type":"ContainerStarted","Data":"64da4931044bb8969143de7934333168590cfd0e51dab797d23d4f75cae61e3a"}
Jul 12 17:58:52.880929 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:52.880889    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g" event={"ID":"672bf4e7-8189-4f59-b53d-7e6f792772e6","Type":"ContainerStarted","Data":"b28ec9ba582d0fd458c677dd76b87596930af0c2d6d9c6c272c1e0947c76627f"}
Jul 12 17:58:52.888338 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:52.888299    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z4zjw" event={"ID":"3950f4fb-1ab0-4d37-9954-c12238bcac2c","Type":"ContainerStarted","Data":"26336dc16d6fb3404d9d518cee1a8e9fa567de123a7df31577e46cee4416aec5"}
Jul 12 17:58:52.902280 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:52.902225    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8hn8g" podStartSLOduration=2.536351224 podStartE2EDuration="6.902205916s" totalImagesPullingTime="4.365854692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:58:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:58:52.899986944 +0000 UTC m=+308.100106617" watchObservedRunningTime="2026-07-12 17:58:52.902205916 +0000 UTC m=+308.102325575"
Jul 12 17:58:52.919866 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:58:52.919813    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-z4zjw" podStartSLOduration=1.8656392830000001 podStartE2EDuration="5.91979523s" totalImagesPullingTime="4.054155947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:58:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:58:52.917667223 +0000 UTC m=+308.117786878" watchObservedRunningTime="2026-07-12 17:58:52.91979523 +0000 UTC m=+308.119914886"
Jul 12 17:59:11.658276 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.658242    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"]
Jul 12 17:59:11.663599 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.663580    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:11.666133 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.666106    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 12 17:59:11.667056 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.667037    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 12 17:59:11.667213 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.667045    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 12 17:59:11.667290 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.667098    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2kpkp\""
Jul 12 17:59:11.671932 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.671913    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"]
Jul 12 17:59:11.787394 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.787368    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c42b9eb8-29a5-4bcd-9c14-d4afb2da0538-metrics-cert\") pod \"lws-controller-manager-5484cf687f-5lnzc\" (UID: \"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:11.787535 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.787430    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c42b9eb8-29a5-4bcd-9c14-d4afb2da0538-cert\") pod \"lws-controller-manager-5484cf687f-5lnzc\" (UID: \"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:11.787535 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.787472    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c42b9eb8-29a5-4bcd-9c14-d4afb2da0538-manager-config\") pod \"lws-controller-manager-5484cf687f-5lnzc\" (UID: \"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:11.787535 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.787506    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw6dv\" (UniqueName: \"kubernetes.io/projected/c42b9eb8-29a5-4bcd-9c14-d4afb2da0538-kube-api-access-lw6dv\") pod \"lws-controller-manager-5484cf687f-5lnzc\" (UID: \"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:11.888238 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.888210    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c42b9eb8-29a5-4bcd-9c14-d4afb2da0538-manager-config\") pod \"lws-controller-manager-5484cf687f-5lnzc\" (UID: \"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:11.888361 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.888254    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw6dv\" (UniqueName: \"kubernetes.io/projected/c42b9eb8-29a5-4bcd-9c14-d4afb2da0538-kube-api-access-lw6dv\") pod \"lws-controller-manager-5484cf687f-5lnzc\" (UID: \"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:11.888361 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.888293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c42b9eb8-29a5-4bcd-9c14-d4afb2da0538-metrics-cert\") pod \"lws-controller-manager-5484cf687f-5lnzc\" (UID: \"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:11.888361 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.888337    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c42b9eb8-29a5-4bcd-9c14-d4afb2da0538-cert\") pod \"lws-controller-manager-5484cf687f-5lnzc\" (UID: \"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:11.888887 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.888857    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c42b9eb8-29a5-4bcd-9c14-d4afb2da0538-manager-config\") pod \"lws-controller-manager-5484cf687f-5lnzc\" (UID: \"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:11.890838 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.890815    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c42b9eb8-29a5-4bcd-9c14-d4afb2da0538-metrics-cert\") pod \"lws-controller-manager-5484cf687f-5lnzc\" (UID: \"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:11.890925 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.890870    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c42b9eb8-29a5-4bcd-9c14-d4afb2da0538-cert\") pod \"lws-controller-manager-5484cf687f-5lnzc\" (UID: \"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:11.899475 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.899455    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw6dv\" (UniqueName: \"kubernetes.io/projected/c42b9eb8-29a5-4bcd-9c14-d4afb2da0538-kube-api-access-lw6dv\") pod \"lws-controller-manager-5484cf687f-5lnzc\" (UID: \"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:11.973278 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:11.973256    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:12.096495 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.096473    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"]
Jul 12 17:59:12.098771 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:59:12.098740    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42b9eb8_29a5_4bcd_9c14_d4afb2da0538.slice/crio-6f56783bb7904b55c96b8ac91b781e8853545d39ef68e74dd1bed7fa6879873b WatchSource:0}: Error finding container 6f56783bb7904b55c96b8ac91b781e8853545d39ef68e74dd1bed7fa6879873b: Status 404 returned error can't find the container with id 6f56783bb7904b55c96b8ac91b781e8853545d39ef68e74dd1bed7fa6879873b
Jul 12 17:59:12.382866 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.382788    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"]
Jul 12 17:59:12.387637 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.387620    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.391728 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.391705    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 12 17:59:12.391824 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.391774    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 12 17:59:12.392005 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.391990    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 12 17:59:12.392455 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.392436    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 12 17:59:12.392697 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.392679    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-m5hgb\""
Jul 12 17:59:12.412771 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.412749    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"]
Jul 12 17:59:12.492902 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.492871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bd5a084-566d-436d-a7af-d65fd1c64ceb-tmp\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.492902 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.492902    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2bd5a084-566d-436d-a7af-d65fd1c64ceb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.493093 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.492936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nw2v\" (UniqueName: \"kubernetes.io/projected/2bd5a084-566d-436d-a7af-d65fd1c64ceb-kube-api-access-9nw2v\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.493093 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.493004    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2bd5a084-566d-436d-a7af-d65fd1c64ceb-manifests\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.493093 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.493054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bd5a084-566d-436d-a7af-d65fd1c64ceb-webhook-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.594359 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.594329    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bd5a084-566d-436d-a7af-d65fd1c64ceb-webhook-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.594502 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.594372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bd5a084-566d-436d-a7af-d65fd1c64ceb-tmp\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.594502 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.594486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2bd5a084-566d-436d-a7af-d65fd1c64ceb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.594605 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.594570    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nw2v\" (UniqueName: \"kubernetes.io/projected/2bd5a084-566d-436d-a7af-d65fd1c64ceb-kube-api-access-9nw2v\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.594605 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.594592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2bd5a084-566d-436d-a7af-d65fd1c64ceb-manifests\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.594937 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.594918    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2bd5a084-566d-436d-a7af-d65fd1c64ceb-manifests\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.596752 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.596730    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bd5a084-566d-436d-a7af-d65fd1c64ceb-tmp\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.596885 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.596865    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bd5a084-566d-436d-a7af-d65fd1c64ceb-webhook-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.596938 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.596882    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2bd5a084-566d-436d-a7af-d65fd1c64ceb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.602934 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.602910    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nw2v\" (UniqueName: \"kubernetes.io/projected/2bd5a084-566d-436d-a7af-d65fd1c64ceb-kube-api-access-9nw2v\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv\" (UID: \"2bd5a084-566d-436d-a7af-d65fd1c64ceb\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.698082 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.698059    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:12.832637 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.832608    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"]
Jul 12 17:59:12.835790 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:59:12.835762    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd5a084_566d_436d_a7af_d65fd1c64ceb.slice/crio-cddc8fa5805e673035c6705d71dab6ba877009a2d621e15935951708330bd493 WatchSource:0}: Error finding container cddc8fa5805e673035c6705d71dab6ba877009a2d621e15935951708330bd493: Status 404 returned error can't find the container with id cddc8fa5805e673035c6705d71dab6ba877009a2d621e15935951708330bd493
Jul 12 17:59:12.969963 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.969890    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv" event={"ID":"2bd5a084-566d-436d-a7af-d65fd1c64ceb","Type":"ContainerStarted","Data":"cddc8fa5805e673035c6705d71dab6ba877009a2d621e15935951708330bd493"}
Jul 12 17:59:12.971030 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:12.971002    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc" event={"ID":"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538","Type":"ContainerStarted","Data":"6f56783bb7904b55c96b8ac91b781e8853545d39ef68e74dd1bed7fa6879873b"}
Jul 12 17:59:16.993678 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:16.993638    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc" event={"ID":"c42b9eb8-29a5-4bcd-9c14-d4afb2da0538","Type":"ContainerStarted","Data":"30aa6a7802b4d6fb496ca82beafc6213e8b8c8e6b413fb6a6136b7331a8f27bf"}
Jul 12 17:59:16.994107 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:16.993737    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:16.995087 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:16.995061    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv" event={"ID":"2bd5a084-566d-436d-a7af-d65fd1c64ceb","Type":"ContainerStarted","Data":"be68abef303bb53808190a007b777382cc8de8bd53a396c87bf2920b8a9e470f"}
Jul 12 17:59:17.011437 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:17.011385    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc" podStartSLOduration=1.781585382 podStartE2EDuration="6.011371387s" totalImagesPullingTime="4.229786005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:17.010250512 +0000 UTC m=+332.210370167" watchObservedRunningTime="2026-07-12 17:59:17.011371387 +0000 UTC m=+332.211491041"
Jul 12 17:59:18.000947 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:18.000916    2578 generic.go:356] "Generic (PLEG): container finished" podID="2bd5a084-566d-436d-a7af-d65fd1c64ceb" containerID="be68abef303bb53808190a007b777382cc8de8bd53a396c87bf2920b8a9e470f" exitCode=0
Jul 12 17:59:18.001375 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:18.001013    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv" event={"ID":"2bd5a084-566d-436d-a7af-d65fd1c64ceb","Type":"ContainerDied","Data":"be68abef303bb53808190a007b777382cc8de8bd53a396c87bf2920b8a9e470f"}
Jul 12 17:59:19.008066 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:19.008026    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv" event={"ID":"2bd5a084-566d-436d-a7af-d65fd1c64ceb","Type":"ContainerStarted","Data":"32cbe93f042d1359bf684845a19be6438b20c26c67ab239c70e0507c750dae7e"}
Jul 12 17:59:19.008451 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:19.008117    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:19.030117 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:19.030070    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv" podStartSLOduration=2.030615334 podStartE2EDuration="7.030054328s" totalImagesPullingTime="3.867474781s" totalInitContainerRuntime="1.131964213s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:19.027586654 +0000 UTC m=+334.227706336" watchObservedRunningTime="2026-07-12 17:59:19.030054328 +0000 UTC m=+334.230174050"
Jul 12 17:59:28.004144 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:28.004109    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-5lnzc"
Jul 12 17:59:30.014701 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:30.014662    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv"
Jul 12 17:59:32.089503 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.089467    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"]
Jul 12 17:59:32.097949 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.097926    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"
Jul 12 17:59:32.101441 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.101422    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 12 17:59:32.101862 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.101846    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 12 17:59:32.102029 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.102005    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-lqljj\""
Jul 12 17:59:32.102029 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.102018    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 12 17:59:32.102929 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.102914    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 12 17:59:32.115894 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.115874    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"]
Jul 12 17:59:32.255178 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.255144    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50471198-3766-43f5-acf6-f7f859a103fd-tmp\") pod \"kube-auth-proxy-795d67df6d-rtl29\" (UID: \"50471198-3766-43f5-acf6-f7f859a103fd\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"
Jul 12 17:59:32.255178 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.255179    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50471198-3766-43f5-acf6-f7f859a103fd-tls-certs\") pod \"kube-auth-proxy-795d67df6d-rtl29\" (UID: \"50471198-3766-43f5-acf6-f7f859a103fd\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"
Jul 12 17:59:32.255413 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.255207    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvphw\" (UniqueName: \"kubernetes.io/projected/50471198-3766-43f5-acf6-f7f859a103fd-kube-api-access-wvphw\") pod \"kube-auth-proxy-795d67df6d-rtl29\" (UID: \"50471198-3766-43f5-acf6-f7f859a103fd\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"
Jul 12 17:59:32.356645 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.356558    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50471198-3766-43f5-acf6-f7f859a103fd-tmp\") pod \"kube-auth-proxy-795d67df6d-rtl29\" (UID: \"50471198-3766-43f5-acf6-f7f859a103fd\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"
Jul 12 17:59:32.356645 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.356597    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50471198-3766-43f5-acf6-f7f859a103fd-tls-certs\") pod \"kube-auth-proxy-795d67df6d-rtl29\" (UID: \"50471198-3766-43f5-acf6-f7f859a103fd\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"
Jul 12 17:59:32.356862 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.356735    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvphw\" (UniqueName: \"kubernetes.io/projected/50471198-3766-43f5-acf6-f7f859a103fd-kube-api-access-wvphw\") pod \"kube-auth-proxy-795d67df6d-rtl29\" (UID: \"50471198-3766-43f5-acf6-f7f859a103fd\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"
Jul 12 17:59:32.359088 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.359060    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50471198-3766-43f5-acf6-f7f859a103fd-tmp\") pod \"kube-auth-proxy-795d67df6d-rtl29\" (UID: \"50471198-3766-43f5-acf6-f7f859a103fd\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"
Jul 12 17:59:32.359271 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.359251    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50471198-3766-43f5-acf6-f7f859a103fd-tls-certs\") pod \"kube-auth-proxy-795d67df6d-rtl29\" (UID: \"50471198-3766-43f5-acf6-f7f859a103fd\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"
Jul 12 17:59:32.374323 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.374299    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvphw\" (UniqueName: \"kubernetes.io/projected/50471198-3766-43f5-acf6-f7f859a103fd-kube-api-access-wvphw\") pod \"kube-auth-proxy-795d67df6d-rtl29\" (UID: \"50471198-3766-43f5-acf6-f7f859a103fd\") " pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"
Jul 12 17:59:32.407196 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.407168    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"
Jul 12 17:59:32.535090 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:32.535066    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-795d67df6d-rtl29"]
Jul 12 17:59:32.537146 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:59:32.537114    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50471198_3766_43f5_acf6_f7f859a103fd.slice/crio-522c8f629a0afd4dc5111ac93666fbdcc1969cf337f9c6cbe5e0c720a3c01df1 WatchSource:0}: Error finding container 522c8f629a0afd4dc5111ac93666fbdcc1969cf337f9c6cbe5e0c720a3c01df1: Status 404 returned error can't find the container with id 522c8f629a0afd4dc5111ac93666fbdcc1969cf337f9c6cbe5e0c720a3c01df1
Jul 12 17:59:33.075267 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:33.075229    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29" event={"ID":"50471198-3766-43f5-acf6-f7f859a103fd","Type":"ContainerStarted","Data":"522c8f629a0afd4dc5111ac93666fbdcc1969cf337f9c6cbe5e0c720a3c01df1"}
Jul 12 17:59:35.086707 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:35.086666    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29" event={"ID":"50471198-3766-43f5-acf6-f7f859a103fd","Type":"ContainerStarted","Data":"6c22660ded75edab6db2dce4b6b068b8349c225eb1fad72b14dc953afc542790"}
Jul 12 17:59:35.103163 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:35.103122    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-795d67df6d-rtl29" podStartSLOduration=0.719218003 podStartE2EDuration="3.10310818s" totalImagesPullingTime="2.383890177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:35.101273488 +0000 UTC m=+350.301393144" watchObservedRunningTime="2026-07-12 17:59:35.10310818 +0000 UTC m=+350.303227866"
Jul 12 17:59:42.012006 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:42.011974    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh"]
Jul 12 17:59:42.015376 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:42.015358    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh"
Jul 12 17:59:42.018036 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:42.018013    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 12 17:59:42.018138 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:42.018015    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wmz7m\""
Jul 12 17:59:42.025872 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:42.025852    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh"]
Jul 12 17:59:42.131839 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:42.131810    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df9zk\" (UniqueName: \"kubernetes.io/projected/206a48fd-5615-432e-8d3c-f23279b775e9-kube-api-access-df9zk\") pod \"llmisvc-controller-manager-777c6bd7d5-tpvlh\" (UID: \"206a48fd-5615-432e-8d3c-f23279b775e9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh"
Jul 12 17:59:42.131976 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:42.131858    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/206a48fd-5615-432e-8d3c-f23279b775e9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tpvlh\" (UID: \"206a48fd-5615-432e-8d3c-f23279b775e9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh"
Jul 12 17:59:42.233030 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:42.232996    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df9zk\" (UniqueName: \"kubernetes.io/projected/206a48fd-5615-432e-8d3c-f23279b775e9-kube-api-access-df9zk\") pod \"llmisvc-controller-manager-777c6bd7d5-tpvlh\" (UID: \"206a48fd-5615-432e-8d3c-f23279b775e9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh"
Jul 12 17:59:42.233159 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:42.233071    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/206a48fd-5615-432e-8d3c-f23279b775e9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tpvlh\" (UID: \"206a48fd-5615-432e-8d3c-f23279b775e9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh"
Jul 12 17:59:42.235653 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:42.235623    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/206a48fd-5615-432e-8d3c-f23279b775e9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tpvlh\" (UID: \"206a48fd-5615-432e-8d3c-f23279b775e9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh"
Jul 12 17:59:42.244996 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:42.244976    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df9zk\" (UniqueName: \"kubernetes.io/projected/206a48fd-5615-432e-8d3c-f23279b775e9-kube-api-access-df9zk\") pod \"llmisvc-controller-manager-777c6bd7d5-tpvlh\" (UID: \"206a48fd-5615-432e-8d3c-f23279b775e9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh"
Jul 12 17:59:42.325436 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:42.325383    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh"
Jul 12 17:59:42.475997 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:42.475970    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh"]
Jul 12 17:59:42.478240 ip-10-0-131-122 kubenswrapper[2578]: W0712 17:59:42.478214    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod206a48fd_5615_432e_8d3c_f23279b775e9.slice/crio-d785b4166ccb1f80f254333845c672d48270cba8523b74fa38fc1141d19330b4 WatchSource:0}: Error finding container d785b4166ccb1f80f254333845c672d48270cba8523b74fa38fc1141d19330b4: Status 404 returned error can't find the container with id d785b4166ccb1f80f254333845c672d48270cba8523b74fa38fc1141d19330b4
Jul 12 17:59:43.123202 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:43.123168    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh" event={"ID":"206a48fd-5615-432e-8d3c-f23279b775e9","Type":"ContainerStarted","Data":"d785b4166ccb1f80f254333845c672d48270cba8523b74fa38fc1141d19330b4"}
Jul 12 17:59:46.142889 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:46.142854    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh" event={"ID":"206a48fd-5615-432e-8d3c-f23279b775e9","Type":"ContainerStarted","Data":"593f62b06e9aec17defc5adde5dc437b486cdaccf7a2bd825ad71aa11fc1dff7"}
Jul 12 17:59:46.143276 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:46.143001    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh"
Jul 12 17:59:46.158240 ip-10-0-131-122 kubenswrapper[2578]: I0712 17:59:46.158196    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh" podStartSLOduration=2.605723858 podStartE2EDuration="5.158183874s" totalImagesPullingTime="2.552460016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:46.157021046 +0000 UTC m=+361.357140718" watchObservedRunningTime="2026-07-12 17:59:46.158183874 +0000 UTC m=+361.358303529"
Jul 12 18:00:17.164222 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:00:17.164192    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tpvlh"
Jul 12 18:01:12.281305 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.281233    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-656f5c87fb-hptk4"]
Jul 12 18:01:12.284612 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.284595    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.287464 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.287441    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 12 18:01:12.288355 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.288341    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 12 18:01:12.288455 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.288389    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 12 18:01:12.288455 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.288410    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 12 18:01:12.288627 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.288550    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 12 18:01:12.288627 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.288586    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 12 18:01:12.288732 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.288702    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4rmzj\""
Jul 12 18:01:12.289273 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.289254    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 12 18:01:12.293676 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.293658    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 12 18:01:12.297546 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.297511    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-656f5c87fb-hptk4"]
Jul 12 18:01:12.344869 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.344850    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-console-config\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.344977 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.344894    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-trusted-ca-bundle\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.344977 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.344917    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hng5z\" (UniqueName: \"kubernetes.io/projected/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-kube-api-access-hng5z\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.344977 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.344959    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-service-ca\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.344977 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.344976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-console-serving-cert\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.345191 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.344997    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-oauth-serving-cert\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.345191 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.345054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-console-oauth-config\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.446233 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.446206    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-service-ca\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.446337 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.446241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-console-serving-cert\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.446337 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.446264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-oauth-serving-cert\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.446337 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.446326    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-console-oauth-config\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.446492 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.446386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-console-config\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.446492 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.446430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-trusted-ca-bundle\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.446636 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.446463    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hng5z\" (UniqueName: \"kubernetes.io/projected/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-kube-api-access-hng5z\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.447029 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.447007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-service-ca\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.447121 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.447071    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-console-config\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.447187 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.447119    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-oauth-serving-cert\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.447277 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.447258    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-trusted-ca-bundle\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.448821 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.448791    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-console-serving-cert\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.448912 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.448866    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-console-oauth-config\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.459402 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.459381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hng5z\" (UniqueName: \"kubernetes.io/projected/4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc-kube-api-access-hng5z\") pod \"console-656f5c87fb-hptk4\" (UID: \"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc\") " pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.594743 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.594680    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:12.727853 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:12.727832    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-656f5c87fb-hptk4"]
Jul 12 18:01:12.730127 ip-10-0-131-122 kubenswrapper[2578]: W0712 18:01:12.730100    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4917a3e2_1aaf_4e6f_bbc0_17f1dc8f72dc.slice/crio-434a0b6bd77c3953e32d3acfc3392aef9d9b1133d6bd943d5cc09e2100ab55bc WatchSource:0}: Error finding container 434a0b6bd77c3953e32d3acfc3392aef9d9b1133d6bd943d5cc09e2100ab55bc: Status 404 returned error can't find the container with id 434a0b6bd77c3953e32d3acfc3392aef9d9b1133d6bd943d5cc09e2100ab55bc
Jul 12 18:01:13.551203 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:13.551167    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656f5c87fb-hptk4" event={"ID":"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc","Type":"ContainerStarted","Data":"3c84cee377048c77581d38fffde45aa0ab9f66bcc6e79b10ae1aa344a886b9bb"}
Jul 12 18:01:13.551574 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:13.551222    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656f5c87fb-hptk4" event={"ID":"4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc","Type":"ContainerStarted","Data":"434a0b6bd77c3953e32d3acfc3392aef9d9b1133d6bd943d5cc09e2100ab55bc"}
Jul 12 18:01:13.569350 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:13.569303    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-656f5c87fb-hptk4" podStartSLOduration=1.569288462 podStartE2EDuration="1.569288462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:01:13.567811136 +0000 UTC m=+448.767930807" watchObservedRunningTime="2026-07-12 18:01:13.569288462 +0000 UTC m=+448.769408117"
Jul 12 18:01:22.594806 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:22.594771    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:22.595376 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:22.594841    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:22.599310 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:22.599286    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:23.606715 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:23.606677    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-656f5c87fb-hptk4"
Jul 12 18:01:26.056033 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.056001    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"]
Jul 12 18:01:26.059814 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.059795    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"
Jul 12 18:01:26.062375 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.062350    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 12 18:01:26.063384 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.063336    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 12 18:01:26.063384 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.063341    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 12 18:01:26.063384 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.063372    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6bjmk\""
Jul 12 18:01:26.063661 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.063343    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 12 18:01:26.065656 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.065634    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"]
Jul 12 18:01:26.160812 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.160787    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6a08f283-ffe2-4a00-9d5a-4b3c28a483cf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xxnzw\" (UID: \"6a08f283-ffe2-4a00-9d5a-4b3c28a483cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"
Jul 12 18:01:26.160947 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.160834    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a08f283-ffe2-4a00-9d5a-4b3c28a483cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xxnzw\" (UID: \"6a08f283-ffe2-4a00-9d5a-4b3c28a483cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"
Jul 12 18:01:26.160947 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.160868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrh8h\" (UniqueName: \"kubernetes.io/projected/6a08f283-ffe2-4a00-9d5a-4b3c28a483cf-kube-api-access-hrh8h\") pod \"kuadrant-console-plugin-66b5774985-xxnzw\" (UID: \"6a08f283-ffe2-4a00-9d5a-4b3c28a483cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"
Jul 12 18:01:26.261651 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.261627    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrh8h\" (UniqueName: \"kubernetes.io/projected/6a08f283-ffe2-4a00-9d5a-4b3c28a483cf-kube-api-access-hrh8h\") pod \"kuadrant-console-plugin-66b5774985-xxnzw\" (UID: \"6a08f283-ffe2-4a00-9d5a-4b3c28a483cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"
Jul 12 18:01:26.261774 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.261684    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6a08f283-ffe2-4a00-9d5a-4b3c28a483cf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xxnzw\" (UID: \"6a08f283-ffe2-4a00-9d5a-4b3c28a483cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"
Jul 12 18:01:26.261774 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.261719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a08f283-ffe2-4a00-9d5a-4b3c28a483cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xxnzw\" (UID: \"6a08f283-ffe2-4a00-9d5a-4b3c28a483cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"
Jul 12 18:01:26.261883 ip-10-0-131-122 kubenswrapper[2578]: E0712 18:01:26.261813    2578 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 12 18:01:26.261883 ip-10-0-131-122 kubenswrapper[2578]: E0712 18:01:26.261869    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a08f283-ffe2-4a00-9d5a-4b3c28a483cf-plugin-serving-cert podName:6a08f283-ffe2-4a00-9d5a-4b3c28a483cf nodeName:}" failed. No retries permitted until 2026-07-12 18:01:26.7618499 +0000 UTC m=+461.961969549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/6a08f283-ffe2-4a00-9d5a-4b3c28a483cf-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-xxnzw" (UID: "6a08f283-ffe2-4a00-9d5a-4b3c28a483cf") : secret "plugin-serving-cert" not found
Jul 12 18:01:26.262400 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.262381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6a08f283-ffe2-4a00-9d5a-4b3c28a483cf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xxnzw\" (UID: \"6a08f283-ffe2-4a00-9d5a-4b3c28a483cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"
Jul 12 18:01:26.270424 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.270397    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrh8h\" (UniqueName: \"kubernetes.io/projected/6a08f283-ffe2-4a00-9d5a-4b3c28a483cf-kube-api-access-hrh8h\") pod \"kuadrant-console-plugin-66b5774985-xxnzw\" (UID: \"6a08f283-ffe2-4a00-9d5a-4b3c28a483cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"
Jul 12 18:01:26.764910 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.764876    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a08f283-ffe2-4a00-9d5a-4b3c28a483cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xxnzw\" (UID: \"6a08f283-ffe2-4a00-9d5a-4b3c28a483cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"
Jul 12 18:01:26.767213 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.767194    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a08f283-ffe2-4a00-9d5a-4b3c28a483cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xxnzw\" (UID: \"6a08f283-ffe2-4a00-9d5a-4b3c28a483cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"
Jul 12 18:01:26.970810 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:26.970782    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"
Jul 12 18:01:27.097599 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:27.097561    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw"]
Jul 12 18:01:27.101863 ip-10-0-131-122 kubenswrapper[2578]: W0712 18:01:27.101835    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a08f283_ffe2_4a00_9d5a_4b3c28a483cf.slice/crio-0a65f2778c22188203f2c6efc1c206b7b099a04f9c5109be8d22c91c657d23b1 WatchSource:0}: Error finding container 0a65f2778c22188203f2c6efc1c206b7b099a04f9c5109be8d22c91c657d23b1: Status 404 returned error can't find the container with id 0a65f2778c22188203f2c6efc1c206b7b099a04f9c5109be8d22c91c657d23b1
Jul 12 18:01:27.622430 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:27.622393    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw" event={"ID":"6a08f283-ffe2-4a00-9d5a-4b3c28a483cf","Type":"ContainerStarted","Data":"0a65f2778c22188203f2c6efc1c206b7b099a04f9c5109be8d22c91c657d23b1"}
Jul 12 18:01:48.745479 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:48.745438    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw" event={"ID":"6a08f283-ffe2-4a00-9d5a-4b3c28a483cf","Type":"ContainerStarted","Data":"dba550691257c3d9f2e7b10f38fdb9f980db36a716dcc5b531b623bf817b2b1a"}
Jul 12 18:01:48.763336 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:01:48.763290    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xxnzw" podStartSLOduration=1.8066761169999999 podStartE2EDuration="22.763278535s" totalImagesPullingTime="20.956602418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:01:48.761794752 +0000 UTC m=+483.961914407" watchObservedRunningTime="2026-07-12 18:01:48.763278535 +0000 UTC m=+483.963398236"
Jul 12 18:02:24.295445 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.295414    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-8xqtr"]
Jul 12 18:02:24.341240 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.341214    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-8xqtr"]
Jul 12 18:02:24.341387 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.341294    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8xqtr"
Jul 12 18:02:24.343897 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.343869    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bm6zs\""
Jul 12 18:02:24.460907 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.460880    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-dzrrz"]
Jul 12 18:02:24.464227 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.464205    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dzrrz"
Jul 12 18:02:24.470644 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.470616    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dzrrz"]
Jul 12 18:02:24.484014 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.483991    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n44th\" (UniqueName: \"kubernetes.io/projected/528dd9f7-7350-42e6-82f0-d214efb9e15f-kube-api-access-n44th\") pod \"authorino-58cd955675-8xqtr\" (UID: \"528dd9f7-7350-42e6-82f0-d214efb9e15f\") " pod="kuadrant-system/authorino-58cd955675-8xqtr"
Jul 12 18:02:24.585335 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.585279    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh2z5\" (UniqueName: \"kubernetes.io/projected/477c7008-8aa3-4d94-9519-bbde7ce15354-kube-api-access-fh2z5\") pod \"authorino-75d556bd77-dzrrz\" (UID: \"477c7008-8aa3-4d94-9519-bbde7ce15354\") " pod="kuadrant-system/authorino-75d556bd77-dzrrz"
Jul 12 18:02:24.585335 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.585322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n44th\" (UniqueName: \"kubernetes.io/projected/528dd9f7-7350-42e6-82f0-d214efb9e15f-kube-api-access-n44th\") pod \"authorino-58cd955675-8xqtr\" (UID: \"528dd9f7-7350-42e6-82f0-d214efb9e15f\") " pod="kuadrant-system/authorino-58cd955675-8xqtr"
Jul 12 18:02:24.596174 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.596152    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n44th\" (UniqueName: \"kubernetes.io/projected/528dd9f7-7350-42e6-82f0-d214efb9e15f-kube-api-access-n44th\") pod \"authorino-58cd955675-8xqtr\" (UID: \"528dd9f7-7350-42e6-82f0-d214efb9e15f\") " pod="kuadrant-system/authorino-58cd955675-8xqtr"
Jul 12 18:02:24.650721 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.650700    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8xqtr"
Jul 12 18:02:24.685954 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.685927    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh2z5\" (UniqueName: \"kubernetes.io/projected/477c7008-8aa3-4d94-9519-bbde7ce15354-kube-api-access-fh2z5\") pod \"authorino-75d556bd77-dzrrz\" (UID: \"477c7008-8aa3-4d94-9519-bbde7ce15354\") " pod="kuadrant-system/authorino-75d556bd77-dzrrz"
Jul 12 18:02:24.694352 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.694329    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh2z5\" (UniqueName: \"kubernetes.io/projected/477c7008-8aa3-4d94-9519-bbde7ce15354-kube-api-access-fh2z5\") pod \"authorino-75d556bd77-dzrrz\" (UID: \"477c7008-8aa3-4d94-9519-bbde7ce15354\") " pod="kuadrant-system/authorino-75d556bd77-dzrrz"
Jul 12 18:02:24.773742 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.773715    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dzrrz"
Jul 12 18:02:24.780004 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.779985    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-8xqtr"]
Jul 12 18:02:24.783953 ip-10-0-131-122 kubenswrapper[2578]: W0712 18:02:24.783930    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod528dd9f7_7350_42e6_82f0_d214efb9e15f.slice/crio-43829ef3bebbe6ab2d660f38bd0d0e750f99c6c3ad176ae698bf9f05566301b7 WatchSource:0}: Error finding container 43829ef3bebbe6ab2d660f38bd0d0e750f99c6c3ad176ae698bf9f05566301b7: Status 404 returned error can't find the container with id 43829ef3bebbe6ab2d660f38bd0d0e750f99c6c3ad176ae698bf9f05566301b7
Jul 12 18:02:24.904628 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.904596    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dzrrz"]
Jul 12 18:02:24.917784 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:24.917755    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8xqtr" event={"ID":"528dd9f7-7350-42e6-82f0-d214efb9e15f","Type":"ContainerStarted","Data":"43829ef3bebbe6ab2d660f38bd0d0e750f99c6c3ad176ae698bf9f05566301b7"}
Jul 12 18:02:25.926958 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:25.926920    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dzrrz" event={"ID":"477c7008-8aa3-4d94-9519-bbde7ce15354","Type":"ContainerStarted","Data":"a9a67828b5eed710eac41e9a295e4d4c3bfd471d954e63b5091a6465f53f95fe"}
Jul 12 18:02:28.947311 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:28.947273    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8xqtr" event={"ID":"528dd9f7-7350-42e6-82f0-d214efb9e15f","Type":"ContainerStarted","Data":"d1d929fa6a0e418de262711479e50d06c0290c7c6eb3f1f45a2f252602204671"}
Jul 12 18:02:28.948831 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:28.948805    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dzrrz" event={"ID":"477c7008-8aa3-4d94-9519-bbde7ce15354","Type":"ContainerStarted","Data":"45cbdf291e3968ad10e00fc54a8ac9921c6cc818febc0dead20f412aa8118c81"}
Jul 12 18:02:28.964044 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:28.964003    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-8xqtr" podStartSLOduration=1.4133723489999999 podStartE2EDuration="4.963990008s" totalImagesPullingTime="3.550617659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:02:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:02:28.962235034 +0000 UTC m=+524.162354692" watchObservedRunningTime="2026-07-12 18:02:28.963990008 +0000 UTC m=+524.164109690"
Jul 12 18:02:28.976706 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:28.976667    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-dzrrz" podStartSLOduration=1.5400279590000001 podStartE2EDuration="4.976653967s" totalImagesPullingTime="3.436626008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:02:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:02:28.975619615 +0000 UTC m=+524.175739301" watchObservedRunningTime="2026-07-12 18:02:28.976653967 +0000 UTC m=+524.176773622"
Jul 12 18:02:28.998098 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:28.998071    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-8xqtr"]
Jul 12 18:02:30.960129 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:30.960083    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-8xqtr" podUID="528dd9f7-7350-42e6-82f0-d214efb9e15f" containerName="authorino" containerID="cri-o://d1d929fa6a0e418de262711479e50d06c0290c7c6eb3f1f45a2f252602204671" gracePeriod=30
Jul 12 18:02:31.205457 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:31.205432    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8xqtr"
Jul 12 18:02:31.345901 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:31.345839    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n44th\" (UniqueName: \"kubernetes.io/projected/528dd9f7-7350-42e6-82f0-d214efb9e15f-kube-api-access-n44th\") pod \"528dd9f7-7350-42e6-82f0-d214efb9e15f\" (UID: \"528dd9f7-7350-42e6-82f0-d214efb9e15f\") "
Jul 12 18:02:31.347912 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:31.347880    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/528dd9f7-7350-42e6-82f0-d214efb9e15f-kube-api-access-n44th" (OuterVolumeSpecName: "kube-api-access-n44th") pod "528dd9f7-7350-42e6-82f0-d214efb9e15f" (UID: "528dd9f7-7350-42e6-82f0-d214efb9e15f"). InnerVolumeSpecName "kube-api-access-n44th". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:02:31.446465 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:31.446439    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n44th\" (UniqueName: \"kubernetes.io/projected/528dd9f7-7350-42e6-82f0-d214efb9e15f-kube-api-access-n44th\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:02:31.965116 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:31.965085    2578 generic.go:356] "Generic (PLEG): container finished" podID="528dd9f7-7350-42e6-82f0-d214efb9e15f" containerID="d1d929fa6a0e418de262711479e50d06c0290c7c6eb3f1f45a2f252602204671" exitCode=0
Jul 12 18:02:31.965496 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:31.965125    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8xqtr"
Jul 12 18:02:31.965496 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:31.965136    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8xqtr" event={"ID":"528dd9f7-7350-42e6-82f0-d214efb9e15f","Type":"ContainerDied","Data":"d1d929fa6a0e418de262711479e50d06c0290c7c6eb3f1f45a2f252602204671"}
Jul 12 18:02:31.965496 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:31.965167    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8xqtr" event={"ID":"528dd9f7-7350-42e6-82f0-d214efb9e15f","Type":"ContainerDied","Data":"43829ef3bebbe6ab2d660f38bd0d0e750f99c6c3ad176ae698bf9f05566301b7"}
Jul 12 18:02:31.965496 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:31.965182    2578 scope.go:117] "RemoveContainer" containerID="d1d929fa6a0e418de262711479e50d06c0290c7c6eb3f1f45a2f252602204671"
Jul 12 18:02:31.973576 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:31.973558    2578 scope.go:117] "RemoveContainer" containerID="d1d929fa6a0e418de262711479e50d06c0290c7c6eb3f1f45a2f252602204671"
Jul 12 18:02:31.973856 ip-10-0-131-122 kubenswrapper[2578]: E0712 18:02:31.973835    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1d929fa6a0e418de262711479e50d06c0290c7c6eb3f1f45a2f252602204671\": container with ID starting with d1d929fa6a0e418de262711479e50d06c0290c7c6eb3f1f45a2f252602204671 not found: ID does not exist" containerID="d1d929fa6a0e418de262711479e50d06c0290c7c6eb3f1f45a2f252602204671"
Jul 12 18:02:31.973910 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:31.973864    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1d929fa6a0e418de262711479e50d06c0290c7c6eb3f1f45a2f252602204671"} err="failed to get container status \"d1d929fa6a0e418de262711479e50d06c0290c7c6eb3f1f45a2f252602204671\": rpc error: code = NotFound desc = could not find container \"d1d929fa6a0e418de262711479e50d06c0290c7c6eb3f1f45a2f252602204671\": container with ID starting with d1d929fa6a0e418de262711479e50d06c0290c7c6eb3f1f45a2f252602204671 not found: ID does not exist"
Jul 12 18:02:31.984920 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:31.984898    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-8xqtr"]
Jul 12 18:02:31.988653 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:31.988635    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-8xqtr"]
Jul 12 18:02:33.324802 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:33.324767    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="528dd9f7-7350-42e6-82f0-d214efb9e15f" path="/var/lib/kubelet/pods/528dd9f7-7350-42e6-82f0-d214efb9e15f/volumes"
Jul 12 18:02:55.886735 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:55.886702    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n5c6x"]
Jul 12 18:02:55.887193 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:55.887048    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="528dd9f7-7350-42e6-82f0-d214efb9e15f" containerName="authorino"
Jul 12 18:02:55.887193 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:55.887060    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="528dd9f7-7350-42e6-82f0-d214efb9e15f" containerName="authorino"
Jul 12 18:02:55.887193 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:55.887141    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="528dd9f7-7350-42e6-82f0-d214efb9e15f" containerName="authorino"
Jul 12 18:02:55.893001 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:55.892979    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-n5c6x"
Jul 12 18:02:55.897226 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:55.897203    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n5c6x"]
Jul 12 18:02:55.948174 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:55.948147    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5mdq\" (UniqueName: \"kubernetes.io/projected/4a0680da-6a5e-47be-9a3d-560abaf659ec-kube-api-access-w5mdq\") pod \"authorino-5d4b67c446-n5c6x\" (UID: \"4a0680da-6a5e-47be-9a3d-560abaf659ec\") " pod="kuadrant-system/authorino-5d4b67c446-n5c6x"
Jul 12 18:02:56.048676 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:56.048649    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5mdq\" (UniqueName: \"kubernetes.io/projected/4a0680da-6a5e-47be-9a3d-560abaf659ec-kube-api-access-w5mdq\") pod \"authorino-5d4b67c446-n5c6x\" (UID: \"4a0680da-6a5e-47be-9a3d-560abaf659ec\") " pod="kuadrant-system/authorino-5d4b67c446-n5c6x"
Jul 12 18:02:56.056606 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:56.056581    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5mdq\" (UniqueName: \"kubernetes.io/projected/4a0680da-6a5e-47be-9a3d-560abaf659ec-kube-api-access-w5mdq\") pod \"authorino-5d4b67c446-n5c6x\" (UID: \"4a0680da-6a5e-47be-9a3d-560abaf659ec\") " pod="kuadrant-system/authorino-5d4b67c446-n5c6x"
Jul 12 18:02:56.117724 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:56.117697    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n5c6x"]
Jul 12 18:02:56.117912 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:56.117897    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-n5c6x"
Jul 12 18:02:56.245177 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:56.245150    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n5c6x"]
Jul 12 18:02:56.247820 ip-10-0-131-122 kubenswrapper[2578]: W0712 18:02:56.247784    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a0680da_6a5e_47be_9a3d_560abaf659ec.slice/crio-b8f658047dbba928de4436d76e4d7d9606e854516d547de426e95fad5ebb1f61 WatchSource:0}: Error finding container b8f658047dbba928de4436d76e4d7d9606e854516d547de426e95fad5ebb1f61: Status 404 returned error can't find the container with id b8f658047dbba928de4436d76e4d7d9606e854516d547de426e95fad5ebb1f61
Jul 12 18:02:57.082095 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:57.082054    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-n5c6x" event={"ID":"4a0680da-6a5e-47be-9a3d-560abaf659ec","Type":"ContainerStarted","Data":"eb171b94b639f5293e198b3f2053169882ebe55732eda851c080807dad54d559"}
Jul 12 18:02:57.082468 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:57.082102    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-n5c6x" event={"ID":"4a0680da-6a5e-47be-9a3d-560abaf659ec","Type":"ContainerStarted","Data":"b8f658047dbba928de4436d76e4d7d9606e854516d547de426e95fad5ebb1f61"}
Jul 12 18:02:57.082468 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:57.082131    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-n5c6x" podUID="4a0680da-6a5e-47be-9a3d-560abaf659ec" containerName="authorino" containerID="cri-o://eb171b94b639f5293e198b3f2053169882ebe55732eda851c080807dad54d559" gracePeriod=30
Jul 12 18:02:57.096541 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:57.096481    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-n5c6x" podStartSLOduration=1.738344125 podStartE2EDuration="2.096467651s" totalImagesPullingTime="358.123526ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:02:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:02:57.09528297 +0000 UTC m=+552.295402623" watchObservedRunningTime="2026-07-12 18:02:57.096467651 +0000 UTC m=+552.296587307"
Jul 12 18:02:57.328151 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:57.328128    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-n5c6x"
Jul 12 18:02:57.359129 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:57.359067    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5mdq\" (UniqueName: \"kubernetes.io/projected/4a0680da-6a5e-47be-9a3d-560abaf659ec-kube-api-access-w5mdq\") pod \"4a0680da-6a5e-47be-9a3d-560abaf659ec\" (UID: \"4a0680da-6a5e-47be-9a3d-560abaf659ec\") "
Jul 12 18:02:57.361043 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:57.361015    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a0680da-6a5e-47be-9a3d-560abaf659ec-kube-api-access-w5mdq" (OuterVolumeSpecName: "kube-api-access-w5mdq") pod "4a0680da-6a5e-47be-9a3d-560abaf659ec" (UID: "4a0680da-6a5e-47be-9a3d-560abaf659ec"). InnerVolumeSpecName "kube-api-access-w5mdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:02:57.460362 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:57.460335    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5mdq\" (UniqueName: \"kubernetes.io/projected/4a0680da-6a5e-47be-9a3d-560abaf659ec-kube-api-access-w5mdq\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:02:58.087843 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:58.087805    2578 generic.go:356] "Generic (PLEG): container finished" podID="4a0680da-6a5e-47be-9a3d-560abaf659ec" containerID="eb171b94b639f5293e198b3f2053169882ebe55732eda851c080807dad54d559" exitCode=0
Jul 12 18:02:58.087843 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:58.087845    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-n5c6x"
Jul 12 18:02:58.088262 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:58.087894    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-n5c6x" event={"ID":"4a0680da-6a5e-47be-9a3d-560abaf659ec","Type":"ContainerDied","Data":"eb171b94b639f5293e198b3f2053169882ebe55732eda851c080807dad54d559"}
Jul 12 18:02:58.088262 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:58.087936    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-n5c6x" event={"ID":"4a0680da-6a5e-47be-9a3d-560abaf659ec","Type":"ContainerDied","Data":"b8f658047dbba928de4436d76e4d7d9606e854516d547de426e95fad5ebb1f61"}
Jul 12 18:02:58.088262 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:58.087955    2578 scope.go:117] "RemoveContainer" containerID="eb171b94b639f5293e198b3f2053169882ebe55732eda851c080807dad54d559"
Jul 12 18:02:58.096910 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:58.096893    2578 scope.go:117] "RemoveContainer" containerID="eb171b94b639f5293e198b3f2053169882ebe55732eda851c080807dad54d559"
Jul 12 18:02:58.097170 ip-10-0-131-122 kubenswrapper[2578]: E0712 18:02:58.097152    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb171b94b639f5293e198b3f2053169882ebe55732eda851c080807dad54d559\": container with ID starting with eb171b94b639f5293e198b3f2053169882ebe55732eda851c080807dad54d559 not found: ID does not exist" containerID="eb171b94b639f5293e198b3f2053169882ebe55732eda851c080807dad54d559"
Jul 12 18:02:58.097223 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:58.097177    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb171b94b639f5293e198b3f2053169882ebe55732eda851c080807dad54d559"} err="failed to get container status \"eb171b94b639f5293e198b3f2053169882ebe55732eda851c080807dad54d559\": rpc error: code = NotFound desc = could not find container \"eb171b94b639f5293e198b3f2053169882ebe55732eda851c080807dad54d559\": container with ID starting with eb171b94b639f5293e198b3f2053169882ebe55732eda851c080807dad54d559 not found: ID does not exist"
Jul 12 18:02:58.108039 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:58.108014    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n5c6x"]
Jul 12 18:02:58.109984 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:58.109964    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n5c6x"]
Jul 12 18:02:59.324841 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:59.324808    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a0680da-6a5e-47be-9a3d-560abaf659ec" path="/var/lib/kubelet/pods/4a0680da-6a5e-47be-9a3d-560abaf659ec/volumes"
Jul 12 18:02:59.815983 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:59.815947    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dzrrz"]
Jul 12 18:02:59.816205 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:02:59.816145    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-dzrrz" podUID="477c7008-8aa3-4d94-9519-bbde7ce15354" containerName="authorino" containerID="cri-o://45cbdf291e3968ad10e00fc54a8ac9921c6cc818febc0dead20f412aa8118c81" gracePeriod=30
Jul 12 18:03:00.064088 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:00.064063    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dzrrz"
Jul 12 18:03:00.099046 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:00.098977    2578 generic.go:356] "Generic (PLEG): container finished" podID="477c7008-8aa3-4d94-9519-bbde7ce15354" containerID="45cbdf291e3968ad10e00fc54a8ac9921c6cc818febc0dead20f412aa8118c81" exitCode=0
Jul 12 18:03:00.099046 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:00.099008    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dzrrz"
Jul 12 18:03:00.099265 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:00.099052    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dzrrz" event={"ID":"477c7008-8aa3-4d94-9519-bbde7ce15354","Type":"ContainerDied","Data":"45cbdf291e3968ad10e00fc54a8ac9921c6cc818febc0dead20f412aa8118c81"}
Jul 12 18:03:00.099265 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:00.099092    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dzrrz" event={"ID":"477c7008-8aa3-4d94-9519-bbde7ce15354","Type":"ContainerDied","Data":"a9a67828b5eed710eac41e9a295e4d4c3bfd471d954e63b5091a6465f53f95fe"}
Jul 12 18:03:00.099265 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:00.099117    2578 scope.go:117] "RemoveContainer" containerID="45cbdf291e3968ad10e00fc54a8ac9921c6cc818febc0dead20f412aa8118c81"
Jul 12 18:03:00.106934 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:00.106919    2578 scope.go:117] "RemoveContainer" containerID="45cbdf291e3968ad10e00fc54a8ac9921c6cc818febc0dead20f412aa8118c81"
Jul 12 18:03:00.107189 ip-10-0-131-122 kubenswrapper[2578]: E0712 18:03:00.107173    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45cbdf291e3968ad10e00fc54a8ac9921c6cc818febc0dead20f412aa8118c81\": container with ID starting with 45cbdf291e3968ad10e00fc54a8ac9921c6cc818febc0dead20f412aa8118c81 not found: ID does not exist" containerID="45cbdf291e3968ad10e00fc54a8ac9921c6cc818febc0dead20f412aa8118c81"
Jul 12 18:03:00.107237 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:00.107205    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45cbdf291e3968ad10e00fc54a8ac9921c6cc818febc0dead20f412aa8118c81"} err="failed to get container status \"45cbdf291e3968ad10e00fc54a8ac9921c6cc818febc0dead20f412aa8118c81\": rpc error: code = NotFound desc = could not find container \"45cbdf291e3968ad10e00fc54a8ac9921c6cc818febc0dead20f412aa8118c81\": container with ID starting with 45cbdf291e3968ad10e00fc54a8ac9921c6cc818febc0dead20f412aa8118c81 not found: ID does not exist"
Jul 12 18:03:00.181057 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:00.181035    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fh2z5\" (UniqueName: \"kubernetes.io/projected/477c7008-8aa3-4d94-9519-bbde7ce15354-kube-api-access-fh2z5\") pod \"477c7008-8aa3-4d94-9519-bbde7ce15354\" (UID: \"477c7008-8aa3-4d94-9519-bbde7ce15354\") "
Jul 12 18:03:00.182903 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:00.182883    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/477c7008-8aa3-4d94-9519-bbde7ce15354-kube-api-access-fh2z5" (OuterVolumeSpecName: "kube-api-access-fh2z5") pod "477c7008-8aa3-4d94-9519-bbde7ce15354" (UID: "477c7008-8aa3-4d94-9519-bbde7ce15354"). InnerVolumeSpecName "kube-api-access-fh2z5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:03:00.282429 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:00.282407    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fh2z5\" (UniqueName: \"kubernetes.io/projected/477c7008-8aa3-4d94-9519-bbde7ce15354-kube-api-access-fh2z5\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:03:00.418941 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:00.418916    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dzrrz"]
Jul 12 18:03:00.422246 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:00.422219    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dzrrz"]
Jul 12 18:03:01.324781 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:01.324747    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="477c7008-8aa3-4d94-9519-bbde7ce15354" path="/var/lib/kubelet/pods/477c7008-8aa3-4d94-9519-bbde7ce15354/volumes"
Jul 12 18:03:04.961663 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:04.961632    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d9864c598-5n8pv"]
Jul 12 18:03:04.961986 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:04.961969    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="477c7008-8aa3-4d94-9519-bbde7ce15354" containerName="authorino"
Jul 12 18:03:04.961986 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:04.961979    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="477c7008-8aa3-4d94-9519-bbde7ce15354" containerName="authorino"
Jul 12 18:03:04.962063 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:04.961998    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a0680da-6a5e-47be-9a3d-560abaf659ec" containerName="authorino"
Jul 12 18:03:04.962063 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:04.962004    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a0680da-6a5e-47be-9a3d-560abaf659ec" containerName="authorino"
Jul 12 18:03:04.962123 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:04.962065    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="477c7008-8aa3-4d94-9519-bbde7ce15354" containerName="authorino"
Jul 12 18:03:04.962123 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:04.962079    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a0680da-6a5e-47be-9a3d-560abaf659ec" containerName="authorino"
Jul 12 18:03:04.965264 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:04.965246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d9864c598-5n8pv"
Jul 12 18:03:04.968155 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:04.968131    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 12 18:03:04.968259 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:04.968184    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tj8br\""
Jul 12 18:03:04.968405 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:04.968389    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 12 18:03:04.974944 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:04.974923    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d9864c598-5n8pv"]
Jul 12 18:03:05.025555 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:05.025505    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79r8t\" (UniqueName: \"kubernetes.io/projected/6ff83215-3855-473b-b84a-fd7092fc2ac8-kube-api-access-79r8t\") pod \"maas-controller-7d9864c598-5n8pv\" (UID: \"6ff83215-3855-473b-b84a-fd7092fc2ac8\") " pod="opendatahub/maas-controller-7d9864c598-5n8pv"
Jul 12 18:03:05.025685 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:05.025657    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ff83215-3855-473b-b84a-fd7092fc2ac8-webhook-cert\") pod \"maas-controller-7d9864c598-5n8pv\" (UID: \"6ff83215-3855-473b-b84a-fd7092fc2ac8\") " pod="opendatahub/maas-controller-7d9864c598-5n8pv"
Jul 12 18:03:05.126446 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:05.126421    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79r8t\" (UniqueName: \"kubernetes.io/projected/6ff83215-3855-473b-b84a-fd7092fc2ac8-kube-api-access-79r8t\") pod \"maas-controller-7d9864c598-5n8pv\" (UID: \"6ff83215-3855-473b-b84a-fd7092fc2ac8\") " pod="opendatahub/maas-controller-7d9864c598-5n8pv"
Jul 12 18:03:05.126581 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:05.126483    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ff83215-3855-473b-b84a-fd7092fc2ac8-webhook-cert\") pod \"maas-controller-7d9864c598-5n8pv\" (UID: \"6ff83215-3855-473b-b84a-fd7092fc2ac8\") " pod="opendatahub/maas-controller-7d9864c598-5n8pv"
Jul 12 18:03:05.128964 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:05.128875    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ff83215-3855-473b-b84a-fd7092fc2ac8-webhook-cert\") pod \"maas-controller-7d9864c598-5n8pv\" (UID: \"6ff83215-3855-473b-b84a-fd7092fc2ac8\") " pod="opendatahub/maas-controller-7d9864c598-5n8pv"
Jul 12 18:03:05.138039 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:05.138019    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79r8t\" (UniqueName: \"kubernetes.io/projected/6ff83215-3855-473b-b84a-fd7092fc2ac8-kube-api-access-79r8t\") pod \"maas-controller-7d9864c598-5n8pv\" (UID: \"6ff83215-3855-473b-b84a-fd7092fc2ac8\") " pod="opendatahub/maas-controller-7d9864c598-5n8pv"
Jul 12 18:03:05.276918 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:05.276862    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d9864c598-5n8pv"
Jul 12 18:03:05.408982 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:05.408876    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d9864c598-5n8pv"]
Jul 12 18:03:05.411215 ip-10-0-131-122 kubenswrapper[2578]: W0712 18:03:05.411187    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ff83215_3855_473b_b84a_fd7092fc2ac8.slice/crio-8835404ea273e094eab1776d9ca6c6cba651bf31bad1aebabb6335842d58088d WatchSource:0}: Error finding container 8835404ea273e094eab1776d9ca6c6cba651bf31bad1aebabb6335842d58088d: Status 404 returned error can't find the container with id 8835404ea273e094eab1776d9ca6c6cba651bf31bad1aebabb6335842d58088d
Jul 12 18:03:06.130076 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:06.130036    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d9864c598-5n8pv" event={"ID":"6ff83215-3855-473b-b84a-fd7092fc2ac8","Type":"ContainerStarted","Data":"8835404ea273e094eab1776d9ca6c6cba651bf31bad1aebabb6335842d58088d"}
Jul 12 18:03:08.143297 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:08.143260    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d9864c598-5n8pv" event={"ID":"6ff83215-3855-473b-b84a-fd7092fc2ac8","Type":"ContainerStarted","Data":"546da1c241fbb4292b1ba9474bf92ba07d2b050fde4fc14e333ca23cb58912f8"}
Jul 12 18:03:08.143687 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:08.143384    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d9864c598-5n8pv"
Jul 12 18:03:08.161175 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:08.161135    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d9864c598-5n8pv" podStartSLOduration=2.359151708 podStartE2EDuration="4.16112248s" totalImagesPullingTime="1.801970772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:03:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:03:08.160670672 +0000 UTC m=+563.360790346" watchObservedRunningTime="2026-07-12 18:03:08.16112248 +0000 UTC m=+563.361242126"
Jul 12 18:03:18.450529 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:18.450477    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d9864c598-5n8pv"]
Jul 12 18:03:18.450956 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:18.450697    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7d9864c598-5n8pv" podUID="6ff83215-3855-473b-b84a-fd7092fc2ac8" containerName="manager" containerID="cri-o://546da1c241fbb4292b1ba9474bf92ba07d2b050fde4fc14e333ca23cb58912f8" gracePeriod=30
Jul 12 18:03:18.452451 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:18.452424    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d9864c598-5n8pv"
Jul 12 18:03:18.691112 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:18.691083    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d9864c598-5n8pv"
Jul 12 18:03:18.842222 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:18.842152    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79r8t\" (UniqueName: \"kubernetes.io/projected/6ff83215-3855-473b-b84a-fd7092fc2ac8-kube-api-access-79r8t\") pod \"6ff83215-3855-473b-b84a-fd7092fc2ac8\" (UID: \"6ff83215-3855-473b-b84a-fd7092fc2ac8\") "
Jul 12 18:03:18.842222 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:18.842186    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ff83215-3855-473b-b84a-fd7092fc2ac8-webhook-cert\") pod \"6ff83215-3855-473b-b84a-fd7092fc2ac8\" (UID: \"6ff83215-3855-473b-b84a-fd7092fc2ac8\") "
Jul 12 18:03:18.844306 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:18.844280    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ff83215-3855-473b-b84a-fd7092fc2ac8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6ff83215-3855-473b-b84a-fd7092fc2ac8" (UID: "6ff83215-3855-473b-b84a-fd7092fc2ac8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:03:18.844411 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:18.844350    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ff83215-3855-473b-b84a-fd7092fc2ac8-kube-api-access-79r8t" (OuterVolumeSpecName: "kube-api-access-79r8t") pod "6ff83215-3855-473b-b84a-fd7092fc2ac8" (UID: "6ff83215-3855-473b-b84a-fd7092fc2ac8"). InnerVolumeSpecName "kube-api-access-79r8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:03:18.943386 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:18.943365    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79r8t\" (UniqueName: \"kubernetes.io/projected/6ff83215-3855-473b-b84a-fd7092fc2ac8-kube-api-access-79r8t\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:03:18.943386 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:18.943385    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ff83215-3855-473b-b84a-fd7092fc2ac8-webhook-cert\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:03:19.199380 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:19.199348    2578 generic.go:356] "Generic (PLEG): container finished" podID="6ff83215-3855-473b-b84a-fd7092fc2ac8" containerID="546da1c241fbb4292b1ba9474bf92ba07d2b050fde4fc14e333ca23cb58912f8" exitCode=0
Jul 12 18:03:19.199545 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:19.199405    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d9864c598-5n8pv"
Jul 12 18:03:19.199545 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:19.199441    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d9864c598-5n8pv" event={"ID":"6ff83215-3855-473b-b84a-fd7092fc2ac8","Type":"ContainerDied","Data":"546da1c241fbb4292b1ba9474bf92ba07d2b050fde4fc14e333ca23cb58912f8"}
Jul 12 18:03:19.199545 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:19.199493    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d9864c598-5n8pv" event={"ID":"6ff83215-3855-473b-b84a-fd7092fc2ac8","Type":"ContainerDied","Data":"8835404ea273e094eab1776d9ca6c6cba651bf31bad1aebabb6335842d58088d"}
Jul 12 18:03:19.199545 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:19.199532    2578 scope.go:117] "RemoveContainer" containerID="546da1c241fbb4292b1ba9474bf92ba07d2b050fde4fc14e333ca23cb58912f8"
Jul 12 18:03:19.208810 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:19.208793    2578 scope.go:117] "RemoveContainer" containerID="546da1c241fbb4292b1ba9474bf92ba07d2b050fde4fc14e333ca23cb58912f8"
Jul 12 18:03:19.209055 ip-10-0-131-122 kubenswrapper[2578]: E0712 18:03:19.209034    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"546da1c241fbb4292b1ba9474bf92ba07d2b050fde4fc14e333ca23cb58912f8\": container with ID starting with 546da1c241fbb4292b1ba9474bf92ba07d2b050fde4fc14e333ca23cb58912f8 not found: ID does not exist" containerID="546da1c241fbb4292b1ba9474bf92ba07d2b050fde4fc14e333ca23cb58912f8"
Jul 12 18:03:19.209102 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:19.209064    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"546da1c241fbb4292b1ba9474bf92ba07d2b050fde4fc14e333ca23cb58912f8"} err="failed to get container status \"546da1c241fbb4292b1ba9474bf92ba07d2b050fde4fc14e333ca23cb58912f8\": rpc error: code = NotFound desc = could not find container \"546da1c241fbb4292b1ba9474bf92ba07d2b050fde4fc14e333ca23cb58912f8\": container with ID starting with 546da1c241fbb4292b1ba9474bf92ba07d2b050fde4fc14e333ca23cb58912f8 not found: ID does not exist"
Jul 12 18:03:19.220114 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:19.220093    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d9864c598-5n8pv"]
Jul 12 18:03:19.223471 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:19.223455    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d9864c598-5n8pv"]
Jul 12 18:03:19.325130 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:19.325105    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ff83215-3855-473b-b84a-fd7092fc2ac8" path="/var/lib/kubelet/pods/6ff83215-3855-473b-b84a-fd7092fc2ac8/volumes"
Jul 12 18:03:41.010730 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.010696    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-tprz5"]
Jul 12 18:03:41.011125 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.011091    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff83215-3855-473b-b84a-fd7092fc2ac8" containerName="manager"
Jul 12 18:03:41.011125 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.011102    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff83215-3855-473b-b84a-fd7092fc2ac8" containerName="manager"
Jul 12 18:03:41.011194 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.011170    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ff83215-3855-473b-b84a-fd7092fc2ac8" containerName="manager"
Jul 12 18:03:41.014257 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.014239    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-tprz5"
Jul 12 18:03:41.016742 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.016722    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hrk85\""
Jul 12 18:03:41.016859 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.016752    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 12 18:03:41.027378 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.027353    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-tprz5"]
Jul 12 18:03:41.029125 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.029107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7731e1d-da1f-4341-b6ea-436df7790893-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-tprz5\" (UID: \"f7731e1d-da1f-4341-b6ea-436df7790893\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-tprz5"
Jul 12 18:03:41.029183 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.029148    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8858\" (UniqueName: \"kubernetes.io/projected/f7731e1d-da1f-4341-b6ea-436df7790893-kube-api-access-x8858\") pod \"payload-processing-f9b8d9b9b-tprz5\" (UID: \"f7731e1d-da1f-4341-b6ea-436df7790893\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-tprz5"
Jul 12 18:03:41.130017 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.129935    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7731e1d-da1f-4341-b6ea-436df7790893-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-tprz5\" (UID: \"f7731e1d-da1f-4341-b6ea-436df7790893\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-tprz5"
Jul 12 18:03:41.130017 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.130005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8858\" (UniqueName: \"kubernetes.io/projected/f7731e1d-da1f-4341-b6ea-436df7790893-kube-api-access-x8858\") pod \"payload-processing-f9b8d9b9b-tprz5\" (UID: \"f7731e1d-da1f-4341-b6ea-436df7790893\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-tprz5"
Jul 12 18:03:41.130703 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.130679    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7731e1d-da1f-4341-b6ea-436df7790893-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-tprz5\" (UID: \"f7731e1d-da1f-4341-b6ea-436df7790893\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-tprz5"
Jul 12 18:03:41.141688 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.141662    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8858\" (UniqueName: \"kubernetes.io/projected/f7731e1d-da1f-4341-b6ea-436df7790893-kube-api-access-x8858\") pod \"payload-processing-f9b8d9b9b-tprz5\" (UID: \"f7731e1d-da1f-4341-b6ea-436df7790893\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-tprz5"
Jul 12 18:03:41.324080 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.324048    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-tprz5"
Jul 12 18:03:41.452167 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:41.452141    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-tprz5"]
Jul 12 18:03:41.454264 ip-10-0-131-122 kubenswrapper[2578]: W0712 18:03:41.454234    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7731e1d_da1f_4341_b6ea_436df7790893.slice/crio-fc19ddda78e8cacd342a66defae56d8b0f952f1903d51bb0580ab24986d3c0fd WatchSource:0}: Error finding container fc19ddda78e8cacd342a66defae56d8b0f952f1903d51bb0580ab24986d3c0fd: Status 404 returned error can't find the container with id fc19ddda78e8cacd342a66defae56d8b0f952f1903d51bb0580ab24986d3c0fd
Jul 12 18:03:42.307815 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:42.307769    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-tprz5" event={"ID":"f7731e1d-da1f-4341-b6ea-436df7790893","Type":"ContainerStarted","Data":"fc19ddda78e8cacd342a66defae56d8b0f952f1903d51bb0580ab24986d3c0fd"}
Jul 12 18:03:43.313569 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:43.313536    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-tprz5" event={"ID":"f7731e1d-da1f-4341-b6ea-436df7790893","Type":"ContainerStarted","Data":"4738138d50dfb07634e438da9441214a7c40077d11aa98149cd0ce3d1e20796c"}
Jul 12 18:03:43.313919 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:43.313606    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-tprz5"
Jul 12 18:03:43.330051 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:43.329999    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-tprz5" podStartSLOduration=1.988228244 podStartE2EDuration="3.32998593s" totalImagesPullingTime="1.341757686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:03:43.327652904 +0000 UTC m=+598.527772559" watchObservedRunningTime="2026-07-12 18:03:43.32998593 +0000 UTC m=+598.530105584"
Jul 12 18:03:45.196864 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:45.196838    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:03:45.200351 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:45.200331    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:03:54.320410 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:03:54.320372    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-tprz5"
Jul 12 18:04:03.903080 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:03.903045    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"]
Jul 12 18:04:03.906607 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:03.906590    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:03.910213 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:03.910191    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p7s5j\""
Jul 12 18:04:03.910334 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:03.910243    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 12 18:04:03.910334 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:03.910310    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 12 18:04:03.910544 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:03.910530    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 12 18:04:03.916919 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:03.916896    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"]
Jul 12 18:04:04.035911 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.035883    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33165e26-93eb-423b-892b-88d630bbf501-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.036042 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.035915    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33165e26-93eb-423b-892b-88d630bbf501-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.036042 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.035941    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33165e26-93eb-423b-892b-88d630bbf501-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.036042 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.035964    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdptz\" (UniqueName: \"kubernetes.io/projected/33165e26-93eb-423b-892b-88d630bbf501-kube-api-access-vdptz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.036042 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.035996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33165e26-93eb-423b-892b-88d630bbf501-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.036185 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.036070    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33165e26-93eb-423b-892b-88d630bbf501-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.137277 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.137254    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33165e26-93eb-423b-892b-88d630bbf501-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.137385 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.137302    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33165e26-93eb-423b-892b-88d630bbf501-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.137385 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.137327    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33165e26-93eb-423b-892b-88d630bbf501-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.137385 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.137355    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33165e26-93eb-423b-892b-88d630bbf501-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.137385 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.137377    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdptz\" (UniqueName: \"kubernetes.io/projected/33165e26-93eb-423b-892b-88d630bbf501-kube-api-access-vdptz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.137567 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.137413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33165e26-93eb-423b-892b-88d630bbf501-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.137673 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.137652    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33165e26-93eb-423b-892b-88d630bbf501-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.137795 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.137777    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33165e26-93eb-423b-892b-88d630bbf501-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.137870 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.137816    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33165e26-93eb-423b-892b-88d630bbf501-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.139629 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.139606    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33165e26-93eb-423b-892b-88d630bbf501-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.139887 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.139868    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33165e26-93eb-423b-892b-88d630bbf501-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.151480 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.151457    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdptz\" (UniqueName: \"kubernetes.io/projected/33165e26-93eb-423b-892b-88d630bbf501-kube-api-access-vdptz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4\" (UID: \"33165e26-93eb-423b-892b-88d630bbf501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.218354 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.218333    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:04.342660 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.342611    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"]
Jul 12 18:04:04.345502 ip-10-0-131-122 kubenswrapper[2578]: W0712 18:04:04.345468    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33165e26_93eb_423b_892b_88d630bbf501.slice/crio-d402e2a36f6ca4b89491f1e9c53e824c7a2e98bc70e5ad25e210b16e5e36f1b4 WatchSource:0}: Error finding container d402e2a36f6ca4b89491f1e9c53e824c7a2e98bc70e5ad25e210b16e5e36f1b4: Status 404 returned error can't find the container with id d402e2a36f6ca4b89491f1e9c53e824c7a2e98bc70e5ad25e210b16e5e36f1b4
Jul 12 18:04:04.347003 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.346985    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 18:04:04.413449 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:04.413418    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4" event={"ID":"33165e26-93eb-423b-892b-88d630bbf501","Type":"ContainerStarted","Data":"d402e2a36f6ca4b89491f1e9c53e824c7a2e98bc70e5ad25e210b16e5e36f1b4"}
Jul 12 18:04:06.424177 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:06.424146    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4" event={"ID":"33165e26-93eb-423b-892b-88d630bbf501","Type":"ContainerStarted","Data":"4de27a0b52a0bfa33e6e949eca415bb0b88bdce96932a1108010a47f726e6d2d"}
Jul 12 18:04:06.441862 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:06.441815    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4" podStartSLOduration=2.264375342 podStartE2EDuration="3.441797036s" totalImagesPullingTime="1.177421694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:04:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:04:06.440185242 +0000 UTC m=+621.640304920" watchObservedRunningTime="2026-07-12 18:04:06.441797036 +0000 UTC m=+621.641916695"
Jul 12 18:04:14.219115 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:14.219025    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:14.219115 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:14.219097    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:14.231642 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:14.231615    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:14.474606 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:14.474500    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4"
Jul 12 18:04:28.208050 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.208012    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"]
Jul 12 18:04:28.212266 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.212246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.214694 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.214671    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 12 18:04:28.221820 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.221789    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"]
Jul 12 18:04:28.254109 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.254080    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4494d08-aae7-4205-a9a0-baf0e32de15f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.254292 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.254188    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4494d08-aae7-4205-a9a0-baf0e32de15f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.254292 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.254236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h9xt\" (UniqueName: \"kubernetes.io/projected/e4494d08-aae7-4205-a9a0-baf0e32de15f-kube-api-access-5h9xt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.254292 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.254290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4494d08-aae7-4205-a9a0-baf0e32de15f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.254467 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.254309    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4494d08-aae7-4205-a9a0-baf0e32de15f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.254467 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.254325    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4494d08-aae7-4205-a9a0-baf0e32de15f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.355603 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.355565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4494d08-aae7-4205-a9a0-baf0e32de15f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.355777 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.355660    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4494d08-aae7-4205-a9a0-baf0e32de15f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.355777 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.355704    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h9xt\" (UniqueName: \"kubernetes.io/projected/e4494d08-aae7-4205-a9a0-baf0e32de15f-kube-api-access-5h9xt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.355777 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.355772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4494d08-aae7-4205-a9a0-baf0e32de15f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.355897 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.355801    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4494d08-aae7-4205-a9a0-baf0e32de15f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.355897 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.355828    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4494d08-aae7-4205-a9a0-baf0e32de15f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.356172 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.356150    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4494d08-aae7-4205-a9a0-baf0e32de15f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.356247 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.356183    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4494d08-aae7-4205-a9a0-baf0e32de15f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.356247 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.356227    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4494d08-aae7-4205-a9a0-baf0e32de15f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.357959 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.357934    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4494d08-aae7-4205-a9a0-baf0e32de15f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.358253 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.358230    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4494d08-aae7-4205-a9a0-baf0e32de15f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.364374 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.364354    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h9xt\" (UniqueName: \"kubernetes.io/projected/e4494d08-aae7-4205-a9a0-baf0e32de15f-kube-api-access-5h9xt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7\" (UID: \"e4494d08-aae7-4205-a9a0-baf0e32de15f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.522935 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.522851    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:28.657795 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:28.657769    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"]
Jul 12 18:04:28.659741 ip-10-0-131-122 kubenswrapper[2578]: W0712 18:04:28.659714    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4494d08_aae7_4205_a9a0_baf0e32de15f.slice/crio-fec94b213a8e9aeae97956a2663e4d25fd0cebf6ad6588b5947eb71514e6b037 WatchSource:0}: Error finding container fec94b213a8e9aeae97956a2663e4d25fd0cebf6ad6588b5947eb71514e6b037: Status 404 returned error can't find the container with id fec94b213a8e9aeae97956a2663e4d25fd0cebf6ad6588b5947eb71514e6b037
Jul 12 18:04:29.535946 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:29.535913    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7" event={"ID":"e4494d08-aae7-4205-a9a0-baf0e32de15f","Type":"ContainerStarted","Data":"32a34e76918bc7b7cde70384bb8e2fbec349cd5b2ae09eba4e07c4343d3109d3"}
Jul 12 18:04:29.536297 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:29.535968    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7" event={"ID":"e4494d08-aae7-4205-a9a0-baf0e32de15f","Type":"ContainerStarted","Data":"fec94b213a8e9aeae97956a2663e4d25fd0cebf6ad6588b5947eb71514e6b037"}
Jul 12 18:04:29.554820 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:29.554767    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7" podStartSLOduration=1.318389813 podStartE2EDuration="1.554749944s" totalImagesPullingTime="236.360131ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:04:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:04:29.553105296 +0000 UTC m=+644.753224972" watchObservedRunningTime="2026-07-12 18:04:29.554749944 +0000 UTC m=+644.754869600"
Jul 12 18:04:38.523286 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:38.523248    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:38.523706 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:38.523338    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:38.535704 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:38.535684    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:38.591778 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:38.591747    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7"
Jul 12 18:04:51.883127 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:51.883076    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"]
Jul 12 18:04:51.888218 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:51.888197    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:51.890751 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:51.890733    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 12 18:04:51.894868 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:51.894844    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"]
Jul 12 18:04:51.960553 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:51.960502    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:51.960732 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:51.960605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:51.960732 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:51.960638    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxdbn\" (UniqueName: \"kubernetes.io/projected/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-kube-api-access-mxdbn\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:51.960732 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:51.960677    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:51.960732 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:51.960714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:51.960983 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:51.960781    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.061377 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.061344    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.061377 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.061378    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.061603 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.061405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxdbn\" (UniqueName: \"kubernetes.io/projected/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-kube-api-access-mxdbn\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.061603 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.061440    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.061603 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.061481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.061603 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.061569    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.061832 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.061811    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.061899 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.061875    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.061954 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.061898    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.063793 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.063761    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.064076 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.064054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.069614 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.069593    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxdbn\" (UniqueName: \"kubernetes.io/projected/dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc-kube-api-access-mxdbn\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd\" (UID: \"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.198194 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.198171    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:04:52.329367 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.329341    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"]
Jul 12 18:04:52.331486 ip-10-0-131-122 kubenswrapper[2578]: W0712 18:04:52.331461    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddbdeef30_9eb8_4a86_8ae7_5e5c9d07decc.slice/crio-41c81f2ff2ee980acfb9c86e1f5087697bcbf5eadc631c962db63a944bb5d607 WatchSource:0}: Error finding container 41c81f2ff2ee980acfb9c86e1f5087697bcbf5eadc631c962db63a944bb5d607: Status 404 returned error can't find the container with id 41c81f2ff2ee980acfb9c86e1f5087697bcbf5eadc631c962db63a944bb5d607
Jul 12 18:04:52.649973 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.649944    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd" event={"ID":"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc","Type":"ContainerStarted","Data":"61082839272d5fe4d155e1b21b1f5e92f229ab1c65402126c4e87c3180d07c82"}
Jul 12 18:04:52.650117 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.649996    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd" event={"ID":"dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc","Type":"ContainerStarted","Data":"41c81f2ff2ee980acfb9c86e1f5087697bcbf5eadc631c962db63a944bb5d607"}
Jul 12 18:04:52.670352 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:04:52.670307    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd" podStartSLOduration=1.46134607 podStartE2EDuration="1.670293535s" totalImagesPullingTime="208.947465ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:04:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:04:52.668571559 +0000 UTC m=+667.868691214" watchObservedRunningTime="2026-07-12 18:04:52.670293535 +0000 UTC m=+667.870413246"
Jul 12 18:05:02.198702 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:05:02.198669    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:05:02.199105 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:05:02.198714    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:05:02.210901 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:05:02.210875    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:05:02.707755 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:05:02.707723    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd"
Jul 12 18:07:27.424021 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.423943    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5dc7598fc6-txzwt"]
Jul 12 18:07:27.427396 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.427378    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dc7598fc6-txzwt"
Jul 12 18:07:27.431130 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.431101    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 12 18:07:27.431265 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.431133    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 12 18:07:27.431265 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.431108    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tj8br\""
Jul 12 18:07:27.436365 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.436339    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5dc7598fc6-txzwt"]
Jul 12 18:07:27.597829 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.597798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8a6658f-4dfa-4c54-8f78-8155674e1e58-webhook-cert\") pod \"maas-controller-5dc7598fc6-txzwt\" (UID: \"c8a6658f-4dfa-4c54-8f78-8155674e1e58\") " pod="opendatahub/maas-controller-5dc7598fc6-txzwt"
Jul 12 18:07:27.598046 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.597917    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92tcd\" (UniqueName: \"kubernetes.io/projected/c8a6658f-4dfa-4c54-8f78-8155674e1e58-kube-api-access-92tcd\") pod \"maas-controller-5dc7598fc6-txzwt\" (UID: \"c8a6658f-4dfa-4c54-8f78-8155674e1e58\") " pod="opendatahub/maas-controller-5dc7598fc6-txzwt"
Jul 12 18:07:27.698817 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.698790    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92tcd\" (UniqueName: \"kubernetes.io/projected/c8a6658f-4dfa-4c54-8f78-8155674e1e58-kube-api-access-92tcd\") pod \"maas-controller-5dc7598fc6-txzwt\" (UID: \"c8a6658f-4dfa-4c54-8f78-8155674e1e58\") " pod="opendatahub/maas-controller-5dc7598fc6-txzwt"
Jul 12 18:07:27.698975 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.698841    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8a6658f-4dfa-4c54-8f78-8155674e1e58-webhook-cert\") pod \"maas-controller-5dc7598fc6-txzwt\" (UID: \"c8a6658f-4dfa-4c54-8f78-8155674e1e58\") " pod="opendatahub/maas-controller-5dc7598fc6-txzwt"
Jul 12 18:07:27.701250 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.701226    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8a6658f-4dfa-4c54-8f78-8155674e1e58-webhook-cert\") pod \"maas-controller-5dc7598fc6-txzwt\" (UID: \"c8a6658f-4dfa-4c54-8f78-8155674e1e58\") " pod="opendatahub/maas-controller-5dc7598fc6-txzwt"
Jul 12 18:07:27.707615 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.707591    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92tcd\" (UniqueName: \"kubernetes.io/projected/c8a6658f-4dfa-4c54-8f78-8155674e1e58-kube-api-access-92tcd\") pod \"maas-controller-5dc7598fc6-txzwt\" (UID: \"c8a6658f-4dfa-4c54-8f78-8155674e1e58\") " pod="opendatahub/maas-controller-5dc7598fc6-txzwt"
Jul 12 18:07:27.738422 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.738401    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dc7598fc6-txzwt"
Jul 12 18:07:27.871421 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:27.871393    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5dc7598fc6-txzwt"]
Jul 12 18:07:27.872803 ip-10-0-131-122 kubenswrapper[2578]: W0712 18:07:27.872773    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8a6658f_4dfa_4c54_8f78_8155674e1e58.slice/crio-21682f41a33c9cd4e7ad04e4aa552fab9cad4d892dd7ddcc944ee1698a0d68d3 WatchSource:0}: Error finding container 21682f41a33c9cd4e7ad04e4aa552fab9cad4d892dd7ddcc944ee1698a0d68d3: Status 404 returned error can't find the container with id 21682f41a33c9cd4e7ad04e4aa552fab9cad4d892dd7ddcc944ee1698a0d68d3
Jul 12 18:07:28.418976 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:28.418944    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dc7598fc6-txzwt" event={"ID":"c8a6658f-4dfa-4c54-8f78-8155674e1e58","Type":"ContainerStarted","Data":"27b00602891d37cce8dab337300984b4e0b9486e4afae8139ff1f6040ee7f8fb"}
Jul 12 18:07:28.419125 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:28.419022    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dc7598fc6-txzwt" event={"ID":"c8a6658f-4dfa-4c54-8f78-8155674e1e58","Type":"ContainerStarted","Data":"21682f41a33c9cd4e7ad04e4aa552fab9cad4d892dd7ddcc944ee1698a0d68d3"}
Jul 12 18:07:28.419125 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:28.419049    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5dc7598fc6-txzwt"
Jul 12 18:07:28.436640 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:28.436593    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5dc7598fc6-txzwt" podStartSLOduration=1.021142783 podStartE2EDuration="1.436579489s" totalImagesPullingTime="415.436706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:07:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:07:28.433950579 +0000 UTC m=+823.634070234" watchObservedRunningTime="2026-07-12 18:07:28.436579489 +0000 UTC m=+823.636699143"
Jul 12 18:07:39.425225 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:07:39.425189    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5dc7598fc6-txzwt"
Jul 12 18:08:45.230817 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:08:45.230778    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:08:45.231492 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:08:45.231465    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:13:45.266195 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:13:45.266170    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:13:45.268207 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:13:45.268186    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:18:45.299026 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:18:45.298988    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:18:45.299611 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:18:45.299585    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:23:45.334432 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:23:45.334406    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:23:45.336820 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:23:45.336796    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:28:45.373670 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:28:45.373624    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:28:45.377012 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:28:45.376990    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:33:45.405565 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:33:45.405442    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:33:45.409749 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:33:45.409363    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:38:45.436595 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:38:45.436567    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:38:45.441338 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:38:45.441317    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:39:54.404423 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.404383    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"]
Jul 12 18:39:54.408194 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.408178    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.410715 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.410693    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-833acb39\"/\"default-dockercfg-xjnh8\""
Jul 12 18:39:54.411002 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.410984    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-833acb39\"/\"kube-root-ca.crt\""
Jul 12 18:39:54.411561 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.411540    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-833acb39\"/\"openshift-service-ca.crt\""
Jul 12 18:39:54.411659 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.411607    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-833acb39\"/\"e2e-lifecycle-model-833acb39-kserve-self-signed-certs\""
Jul 12 18:39:54.418652 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.418626    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"]
Jul 12 18:39:54.504014 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.503972    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrn24\" (UniqueName: \"kubernetes.io/projected/5d2e2a18-53db-407c-8e2e-c68438d6d633-kube-api-access-wrn24\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.504014 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.504021    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-home\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.504219 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.504113    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-tmp-dir\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.504219 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.504173    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-model-cache\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.504219 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.504203    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-dshm\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.504317 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.504278    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d2e2a18-53db-407c-8e2e-c68438d6d633-tls-certs\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.605691 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.605649    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-tmp-dir\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.605858 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.605710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-model-cache\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.605858 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.605729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-dshm\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.605858 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.605771    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d2e2a18-53db-407c-8e2e-c68438d6d633-tls-certs\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.605858 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.605834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrn24\" (UniqueName: \"kubernetes.io/projected/5d2e2a18-53db-407c-8e2e-c68438d6d633-kube-api-access-wrn24\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.605858 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.605850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-home\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.606113 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.606090    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-tmp-dir\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.606159 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.606112    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-model-cache\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.606244 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.606221    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-home\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.608062 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.608045    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-dshm\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.608339 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.608320    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d2e2a18-53db-407c-8e2e-c68438d6d633-tls-certs\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.614121 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.614096    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrn24\" (UniqueName: \"kubernetes.io/projected/5d2e2a18-53db-407c-8e2e-c68438d6d633-kube-api-access-wrn24\") pod \"e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") " pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.717901 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.717878    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:39:54.853032 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.852999    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"]
Jul 12 18:39:54.855100 ip-10-0-131-122 kubenswrapper[2578]: W0712 18:39:54.855074    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d2e2a18_53db_407c_8e2e_c68438d6d633.slice/crio-11ad4dd19d9b8d9eaea114ff9d907b381824d86477e58693828186e34f357028 WatchSource:0}: Error finding container 11ad4dd19d9b8d9eaea114ff9d907b381824d86477e58693828186e34f357028: Status 404 returned error can't find the container with id 11ad4dd19d9b8d9eaea114ff9d907b381824d86477e58693828186e34f357028
Jul 12 18:39:54.884455 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:54.884431    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm" event={"ID":"5d2e2a18-53db-407c-8e2e-c68438d6d633","Type":"ContainerStarted","Data":"11ad4dd19d9b8d9eaea114ff9d907b381824d86477e58693828186e34f357028"}
Jul 12 18:39:55.890445 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:39:55.890405    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm" event={"ID":"5d2e2a18-53db-407c-8e2e-c68438d6d633","Type":"ContainerStarted","Data":"30d1719e4d4f878cab5e0573c6071b1b5f92ef43034dcb8bb4543c808a97f22b"}
Jul 12 18:40:04.719003 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:40:04.718975    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:40:04.719378 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:40:04.719025    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:40:04.731267 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:40:04.731240    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:40:04.749720 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:40:04.749662    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm" podStartSLOduration=10.749643868 podStartE2EDuration="10.749643868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:39:55.905997369 +0000 UTC m=+2771.106117017" watchObservedRunningTime="2026-07-12 18:40:04.749643868 +0000 UTC m=+2779.949763526"
Jul 12 18:40:04.944935 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:40:04.944900    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:43:45.468171 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:43:45.468066    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:43:45.475535 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:43:45.475495    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:46:31.974089 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:31.974007    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"]
Jul 12 18:46:46.994415 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:46.994369    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm" podUID="5d2e2a18-53db-407c-8e2e-c68438d6d633" containerName="main" containerID="cri-o://30d1719e4d4f878cab5e0573c6071b1b5f92ef43034dcb8bb4543c808a97f22b" gracePeriod=45
Jul 12 18:46:47.247562 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.247499    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:46:47.297882 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.297854    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d2e2a18-53db-407c-8e2e-c68438d6d633-tls-certs\") pod \"5d2e2a18-53db-407c-8e2e-c68438d6d633\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") "
Jul 12 18:46:47.298042 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.297892    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-model-cache\") pod \"5d2e2a18-53db-407c-8e2e-c68438d6d633\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") "
Jul 12 18:46:47.298042 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.297928    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-home\") pod \"5d2e2a18-53db-407c-8e2e-c68438d6d633\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") "
Jul 12 18:46:47.298042 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.297984    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrn24\" (UniqueName: \"kubernetes.io/projected/5d2e2a18-53db-407c-8e2e-c68438d6d633-kube-api-access-wrn24\") pod \"5d2e2a18-53db-407c-8e2e-c68438d6d633\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") "
Jul 12 18:46:47.298212 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.298065    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-dshm\") pod \"5d2e2a18-53db-407c-8e2e-c68438d6d633\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") "
Jul 12 18:46:47.298212 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.298106    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-tmp-dir\") pod \"5d2e2a18-53db-407c-8e2e-c68438d6d633\" (UID: \"5d2e2a18-53db-407c-8e2e-c68438d6d633\") "
Jul 12 18:46:47.298317 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.298260    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-model-cache" (OuterVolumeSpecName: "model-cache") pod "5d2e2a18-53db-407c-8e2e-c68438d6d633" (UID: "5d2e2a18-53db-407c-8e2e-c68438d6d633"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:46:47.298503 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.298473    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-model-cache\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:46:47.298503 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.298487    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5d2e2a18-53db-407c-8e2e-c68438d6d633" (UID: "5d2e2a18-53db-407c-8e2e-c68438d6d633"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:46:47.298671 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.298512    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-home" (OuterVolumeSpecName: "home") pod "5d2e2a18-53db-407c-8e2e-c68438d6d633" (UID: "5d2e2a18-53db-407c-8e2e-c68438d6d633"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:46:47.300373 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.300344    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-dshm" (OuterVolumeSpecName: "dshm") pod "5d2e2a18-53db-407c-8e2e-c68438d6d633" (UID: "5d2e2a18-53db-407c-8e2e-c68438d6d633"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:46:47.300562 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.300543    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d2e2a18-53db-407c-8e2e-c68438d6d633-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5d2e2a18-53db-407c-8e2e-c68438d6d633" (UID: "5d2e2a18-53db-407c-8e2e-c68438d6d633"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:46:47.300635 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.300565    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d2e2a18-53db-407c-8e2e-c68438d6d633-kube-api-access-wrn24" (OuterVolumeSpecName: "kube-api-access-wrn24") pod "5d2e2a18-53db-407c-8e2e-c68438d6d633" (UID: "5d2e2a18-53db-407c-8e2e-c68438d6d633"). InnerVolumeSpecName "kube-api-access-wrn24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:46:47.399740 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.399702    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d2e2a18-53db-407c-8e2e-c68438d6d633-tls-certs\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:46:47.399740 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.399733    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-home\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:46:47.399740 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.399745    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrn24\" (UniqueName: \"kubernetes.io/projected/5d2e2a18-53db-407c-8e2e-c68438d6d633-kube-api-access-wrn24\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:46:47.400030 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.399831    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-dshm\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:46:47.400030 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.399846    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d2e2a18-53db-407c-8e2e-c68438d6d633-tmp-dir\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:46:47.903476 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.903446    2578 generic.go:356] "Generic (PLEG): container finished" podID="5d2e2a18-53db-407c-8e2e-c68438d6d633" containerID="30d1719e4d4f878cab5e0573c6071b1b5f92ef43034dcb8bb4543c808a97f22b" exitCode=0
Jul 12 18:46:47.903664 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.903492    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"
Jul 12 18:46:47.903664 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.903550    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm" event={"ID":"5d2e2a18-53db-407c-8e2e-c68438d6d633","Type":"ContainerDied","Data":"30d1719e4d4f878cab5e0573c6071b1b5f92ef43034dcb8bb4543c808a97f22b"}
Jul 12 18:46:47.903664 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.903592    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm" event={"ID":"5d2e2a18-53db-407c-8e2e-c68438d6d633","Type":"ContainerDied","Data":"11ad4dd19d9b8d9eaea114ff9d907b381824d86477e58693828186e34f357028"}
Jul 12 18:46:47.903664 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.903611    2578 scope.go:117] "RemoveContainer" containerID="30d1719e4d4f878cab5e0573c6071b1b5f92ef43034dcb8bb4543c808a97f22b"
Jul 12 18:46:47.912181 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.912164    2578 scope.go:117] "RemoveContainer" containerID="30d1719e4d4f878cab5e0573c6071b1b5f92ef43034dcb8bb4543c808a97f22b"
Jul 12 18:46:47.912438 ip-10-0-131-122 kubenswrapper[2578]: E0712 18:46:47.912420    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d1719e4d4f878cab5e0573c6071b1b5f92ef43034dcb8bb4543c808a97f22b\": container with ID starting with 30d1719e4d4f878cab5e0573c6071b1b5f92ef43034dcb8bb4543c808a97f22b not found: ID does not exist" containerID="30d1719e4d4f878cab5e0573c6071b1b5f92ef43034dcb8bb4543c808a97f22b"
Jul 12 18:46:47.912500 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.912447    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d1719e4d4f878cab5e0573c6071b1b5f92ef43034dcb8bb4543c808a97f22b"} err="failed to get container status \"30d1719e4d4f878cab5e0573c6071b1b5f92ef43034dcb8bb4543c808a97f22b\": rpc error: code = NotFound desc = could not find container \"30d1719e4d4f878cab5e0573c6071b1b5f92ef43034dcb8bb4543c808a97f22b\": container with ID starting with 30d1719e4d4f878cab5e0573c6071b1b5f92ef43034dcb8bb4543c808a97f22b not found: ID does not exist"
Jul 12 18:46:47.919688 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.919661    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"]
Jul 12 18:46:47.923754 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:47.923734    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-833acb39/e2e-lifecycle-model-833acb39-kserve-7f56f9564c-g8qwm"]
Jul 12 18:46:49.325917 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:46:49.325880    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d2e2a18-53db-407c-8e2e-c68438d6d633" path="/var/lib/kubelet/pods/5d2e2a18-53db-407c-8e2e-c68438d6d633/volumes"
Jul 12 18:48:45.498759 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:48:45.498646    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:48:45.507957 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:48:45.507936    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:49:06.804836 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.804801    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"]
Jul 12 18:49:06.805444 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.805422    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d2e2a18-53db-407c-8e2e-c68438d6d633" containerName="main"
Jul 12 18:49:06.805487 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.805448    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d2e2a18-53db-407c-8e2e-c68438d6d633" containerName="main"
Jul 12 18:49:06.805603 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.805589    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d2e2a18-53db-407c-8e2e-c68438d6d633" containerName="main"
Jul 12 18:49:06.809178 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.809160    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:06.813635 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.813615    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5beeb3\"/\"openshift-service-ca.crt\""
Jul 12 18:49:06.813740 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.813669    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5beeb3\"/\"default-dockercfg-xwjrh\""
Jul 12 18:49:06.814862 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.814822    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5beeb3\"/\"kube-root-ca.crt\""
Jul 12 18:49:06.814975 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.814873    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5beeb3\"/\"test-model-5beeb3-kserve-self-signed-certs\""
Jul 12 18:49:06.821571 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.821501    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"]
Jul 12 18:49:06.924806 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.924780    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-tmp-dir\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:06.924934 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.924812    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-model-cache\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:06.924934 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.924842    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-dshm\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:06.924934 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.924917    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbkwm\" (UniqueName: \"kubernetes.io/projected/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-kube-api-access-vbkwm\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:06.925054 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.924959    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-tls-certs\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:06.925054 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:06.924983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-home\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.026252 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.026182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-tmp-dir\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.026252 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.026216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-model-cache\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.026252 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.026241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-dshm\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.026508 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.026274    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbkwm\" (UniqueName: \"kubernetes.io/projected/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-kube-api-access-vbkwm\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.026508 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.026316    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-tls-certs\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.026508 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.026341    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-home\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.026695 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.026628    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-model-cache\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.026695 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.026648    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-tmp-dir\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.026822 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.026794    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-home\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.028608 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.028578    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-dshm\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.028830 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.028812    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-tls-certs\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.033852 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.033831    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbkwm\" (UniqueName: \"kubernetes.io/projected/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-kube-api-access-vbkwm\") pod \"test-model-5beeb3-kserve-7587c56bcc-rw2kb\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") " pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.118855 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.118795    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:07.250222 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.250051    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"]
Jul 12 18:49:07.252920 ip-10-0-131-122 kubenswrapper[2578]: W0712 18:49:07.252891    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6bf7ae7_14c7_4792_8580_0a4e46ae3a31.slice/crio-05c356c3c3e03019d932241ca914cae6f4a3f80de4eb4a66920e560046ec84a4 WatchSource:0}: Error finding container 05c356c3c3e03019d932241ca914cae6f4a3f80de4eb4a66920e560046ec84a4: Status 404 returned error can't find the container with id 05c356c3c3e03019d932241ca914cae6f4a3f80de4eb4a66920e560046ec84a4
Jul 12 18:49:07.579511 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.579477    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb" event={"ID":"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31","Type":"ContainerStarted","Data":"308cd4c2a36590f5d2e1436705ff6b2f6390cc3cecfb77ba5d5c8d95569e577a"}
Jul 12 18:49:07.579736 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.579545    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb" event={"ID":"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31","Type":"ContainerStarted","Data":"05c356c3c3e03019d932241ca914cae6f4a3f80de4eb4a66920e560046ec84a4"}
Jul 12 18:49:07.602248 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:07.602198    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb" podStartSLOduration=1.6021855459999998 podStartE2EDuration="1.602185546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:49:07.599553646 +0000 UTC m=+3322.799673298" watchObservedRunningTime="2026-07-12 18:49:07.602185546 +0000 UTC m=+3322.802305200"
Jul 12 18:49:17.119338 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:17.119303    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:17.119876 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:17.119587    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:17.132219 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:17.132196    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:17.642420 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:17.642394    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:49:21.998139 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:21.998102    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"]
Jul 12 18:49:22.002734 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.002712    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.005289 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.005261    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d2f6a5\"/\"openshift-service-ca.crt\""
Jul 12 18:49:22.006281 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.006258    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d2f6a5\"/\"kube-root-ca.crt\""
Jul 12 18:49:22.006397 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.006279    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d2f6a5\"/\"default-dockercfg-942g2\""
Jul 12 18:49:22.006397 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.006338    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d2f6a5\"/\"test-model-d2f6a5-kserve-self-signed-certs\""
Jul 12 18:49:22.012065 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.011993    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"]
Jul 12 18:49:22.164209 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.164176    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-home\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.164404 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.164327    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x86l\" (UniqueName: \"kubernetes.io/projected/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-kube-api-access-2x86l\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.164404 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.164361    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-model-cache\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.164404 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.164391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-tls-certs\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.164595 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.164464    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-tmp-dir\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.164595 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.164560    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-dshm\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.265359 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.265290    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x86l\" (UniqueName: \"kubernetes.io/projected/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-kube-api-access-2x86l\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.265359 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.265327    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-model-cache\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.265359 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.265350    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-tls-certs\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.265637 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.265368    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-tmp-dir\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.265637 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.265390    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-dshm\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.265637 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.265423    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-home\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.265797 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.265775    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-model-cache\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.265877 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.265793    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-tmp-dir\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.265877 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.265843    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-home\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.267673 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.267653    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-dshm\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.267983 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.267966    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-tls-certs\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.275508 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.275483    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x86l\" (UniqueName: \"kubernetes.io/projected/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-kube-api-access-2x86l\") pod \"test-model-d2f6a5-kserve-6bd6b48f-zrf6n\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") " pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.315509 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.315485    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:22.442253 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.442227    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"]
Jul 12 18:49:22.444285 ip-10-0-131-122 kubenswrapper[2578]: W0712 18:49:22.444254    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35f778a2_2abb_4bf0_b5af_4226e0aaa5c8.slice/crio-c51f0bbd5585e3403c11c138be4ab9ab820391c61ebbb373b257dabe4dc9645d WatchSource:0}: Error finding container c51f0bbd5585e3403c11c138be4ab9ab820391c61ebbb373b257dabe4dc9645d: Status 404 returned error can't find the container with id c51f0bbd5585e3403c11c138be4ab9ab820391c61ebbb373b257dabe4dc9645d
Jul 12 18:49:22.659186 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.659107    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n" event={"ID":"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8","Type":"ContainerStarted","Data":"6ca443ab341ab006801eb8acf232f9eb23087f00687bffed5669c457c59860eb"}
Jul 12 18:49:22.659186 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.659148    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n" event={"ID":"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8","Type":"ContainerStarted","Data":"c51f0bbd5585e3403c11c138be4ab9ab820391c61ebbb373b257dabe4dc9645d"}
Jul 12 18:49:22.679878 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:22.679815    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n" podStartSLOduration=1.679789687 podStartE2EDuration="1.679789687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:49:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:49:22.676471629 +0000 UTC m=+3337.876591282" watchObservedRunningTime="2026-07-12 18:49:22.679789687 +0000 UTC m=+3337.879909337"
Jul 12 18:49:32.316644 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:32.316606    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:32.317048 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:32.316665    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:32.329138 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:32.329110    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:32.721386 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:32.721354    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:49:44.912702 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:44.912664    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"]
Jul 12 18:49:45.799717 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:45.799685    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"]
Jul 12 18:49:59.933574 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:49:59.933511    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb" podUID="a6bf7ae7-14c7-4792-8580-0a4e46ae3a31" containerName="main" containerID="cri-o://308cd4c2a36590f5d2e1436705ff6b2f6390cc3cecfb77ba5d5c8d95569e577a" gracePeriod=45
Jul 12 18:50:00.183642 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.183587    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:50:00.323832 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.323805    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-model-cache\") pod \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") "
Jul 12 18:50:00.324024 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.323855    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-dshm\") pod \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") "
Jul 12 18:50:00.324024 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.323890    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-tmp-dir\") pod \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") "
Jul 12 18:50:00.324024 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.323921    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-home\") pod \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") "
Jul 12 18:50:00.324024 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.323944    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-tls-certs\") pod \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") "
Jul 12 18:50:00.324024 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.323974    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbkwm\" (UniqueName: \"kubernetes.io/projected/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-kube-api-access-vbkwm\") pod \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\" (UID: \"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31\") "
Jul 12 18:50:00.324275 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.324106    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-model-cache" (OuterVolumeSpecName: "model-cache") pod "a6bf7ae7-14c7-4792-8580-0a4e46ae3a31" (UID: "a6bf7ae7-14c7-4792-8580-0a4e46ae3a31"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:50:00.324275 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.324219    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-home" (OuterVolumeSpecName: "home") pod "a6bf7ae7-14c7-4792-8580-0a4e46ae3a31" (UID: "a6bf7ae7-14c7-4792-8580-0a4e46ae3a31"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:50:00.324275 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.324230    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a6bf7ae7-14c7-4792-8580-0a4e46ae3a31" (UID: "a6bf7ae7-14c7-4792-8580-0a4e46ae3a31"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:50:00.324407 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.324353    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-model-cache\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:50:00.324407 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.324367    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-tmp-dir\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:50:00.324407 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.324375    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-home\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:50:00.326349 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.326327    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-dshm" (OuterVolumeSpecName: "dshm") pod "a6bf7ae7-14c7-4792-8580-0a4e46ae3a31" (UID: "a6bf7ae7-14c7-4792-8580-0a4e46ae3a31"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:50:00.326424 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.326333    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-kube-api-access-vbkwm" (OuterVolumeSpecName: "kube-api-access-vbkwm") pod "a6bf7ae7-14c7-4792-8580-0a4e46ae3a31" (UID: "a6bf7ae7-14c7-4792-8580-0a4e46ae3a31"). InnerVolumeSpecName "kube-api-access-vbkwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:50:00.326424 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.326351    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a6bf7ae7-14c7-4792-8580-0a4e46ae3a31" (UID: "a6bf7ae7-14c7-4792-8580-0a4e46ae3a31"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:50:00.425039 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.425004    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-dshm\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:50:00.425039 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.425033    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-tls-certs\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:50:00.425039 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.425044    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbkwm\" (UniqueName: \"kubernetes.io/projected/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31-kube-api-access-vbkwm\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:50:00.822089 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.822035    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n" podUID="35f778a2-2abb-4bf0-b5af-4226e0aaa5c8" containerName="main" containerID="cri-o://6ca443ab341ab006801eb8acf232f9eb23087f00687bffed5669c457c59860eb" gracePeriod=45
Jul 12 18:50:00.843083 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.843054    2578 generic.go:356] "Generic (PLEG): container finished" podID="a6bf7ae7-14c7-4792-8580-0a4e46ae3a31" containerID="308cd4c2a36590f5d2e1436705ff6b2f6390cc3cecfb77ba5d5c8d95569e577a" exitCode=0
Jul 12 18:50:00.843204 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.843097    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"
Jul 12 18:50:00.843204 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.843129    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb" event={"ID":"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31","Type":"ContainerDied","Data":"308cd4c2a36590f5d2e1436705ff6b2f6390cc3cecfb77ba5d5c8d95569e577a"}
Jul 12 18:50:00.843204 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.843176    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb" event={"ID":"a6bf7ae7-14c7-4792-8580-0a4e46ae3a31","Type":"ContainerDied","Data":"05c356c3c3e03019d932241ca914cae6f4a3f80de4eb4a66920e560046ec84a4"}
Jul 12 18:50:00.843204 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.843193    2578 scope.go:117] "RemoveContainer" containerID="308cd4c2a36590f5d2e1436705ff6b2f6390cc3cecfb77ba5d5c8d95569e577a"
Jul 12 18:50:00.895913 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.895883    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"]
Jul 12 18:50:00.900205 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.900184    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-5beeb3/test-model-5beeb3-kserve-7587c56bcc-rw2kb"]
Jul 12 18:50:00.932228 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.932209    2578 scope.go:117] "RemoveContainer" containerID="308cd4c2a36590f5d2e1436705ff6b2f6390cc3cecfb77ba5d5c8d95569e577a"
Jul 12 18:50:00.932466 ip-10-0-131-122 kubenswrapper[2578]: E0712 18:50:00.932448    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"308cd4c2a36590f5d2e1436705ff6b2f6390cc3cecfb77ba5d5c8d95569e577a\": container with ID starting with 308cd4c2a36590f5d2e1436705ff6b2f6390cc3cecfb77ba5d5c8d95569e577a not found: ID does not exist" containerID="308cd4c2a36590f5d2e1436705ff6b2f6390cc3cecfb77ba5d5c8d95569e577a"
Jul 12 18:50:00.932536 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:00.932473    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"308cd4c2a36590f5d2e1436705ff6b2f6390cc3cecfb77ba5d5c8d95569e577a"} err="failed to get container status \"308cd4c2a36590f5d2e1436705ff6b2f6390cc3cecfb77ba5d5c8d95569e577a\": rpc error: code = NotFound desc = could not find container \"308cd4c2a36590f5d2e1436705ff6b2f6390cc3cecfb77ba5d5c8d95569e577a\": container with ID starting with 308cd4c2a36590f5d2e1436705ff6b2f6390cc3cecfb77ba5d5c8d95569e577a not found: ID does not exist"
Jul 12 18:50:01.070567 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.070544    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:50:01.232559 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.232491    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-home\") pod \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") "
Jul 12 18:50:01.232559 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.232567    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x86l\" (UniqueName: \"kubernetes.io/projected/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-kube-api-access-2x86l\") pod \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") "
Jul 12 18:50:01.232815 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.232602    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-tmp-dir\") pod \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") "
Jul 12 18:50:01.232815 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.232645    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-tls-certs\") pod \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") "
Jul 12 18:50:01.232815 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.232698    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-model-cache\") pod \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") "
Jul 12 18:50:01.232815 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.232722    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-dshm\") pod \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\" (UID: \"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8\") "
Jul 12 18:50:01.232815 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.232758    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-home" (OuterVolumeSpecName: "home") pod "35f778a2-2abb-4bf0-b5af-4226e0aaa5c8" (UID: "35f778a2-2abb-4bf0-b5af-4226e0aaa5c8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:50:01.233041 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.232963    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "35f778a2-2abb-4bf0-b5af-4226e0aaa5c8" (UID: "35f778a2-2abb-4bf0-b5af-4226e0aaa5c8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:50:01.233041 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.232972    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-model-cache" (OuterVolumeSpecName: "model-cache") pod "35f778a2-2abb-4bf0-b5af-4226e0aaa5c8" (UID: "35f778a2-2abb-4bf0-b5af-4226e0aaa5c8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:50:01.233115 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.233077    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-model-cache\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:50:01.233115 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.233091    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-home\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:50:01.233115 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.233099    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-tmp-dir\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:50:01.234912 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.234891    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-kube-api-access-2x86l" (OuterVolumeSpecName: "kube-api-access-2x86l") pod "35f778a2-2abb-4bf0-b5af-4226e0aaa5c8" (UID: "35f778a2-2abb-4bf0-b5af-4226e0aaa5c8"). InnerVolumeSpecName "kube-api-access-2x86l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 18:50:01.234985 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.234896    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-dshm" (OuterVolumeSpecName: "dshm") pod "35f778a2-2abb-4bf0-b5af-4226e0aaa5c8" (UID: "35f778a2-2abb-4bf0-b5af-4226e0aaa5c8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 18:50:01.234985 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.234963    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "35f778a2-2abb-4bf0-b5af-4226e0aaa5c8" (UID: "35f778a2-2abb-4bf0-b5af-4226e0aaa5c8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 18:50:01.327172 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.327141    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6bf7ae7-14c7-4792-8580-0a4e46ae3a31" path="/var/lib/kubelet/pods/a6bf7ae7-14c7-4792-8580-0a4e46ae3a31/volumes"
Jul 12 18:50:01.334426 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.334304    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-dshm\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:50:01.334426 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.334323    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x86l\" (UniqueName: \"kubernetes.io/projected/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-kube-api-access-2x86l\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:50:01.334426 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.334334    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8-tls-certs\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 18:50:01.850692 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.850655    2578 generic.go:356] "Generic (PLEG): container finished" podID="35f778a2-2abb-4bf0-b5af-4226e0aaa5c8" containerID="6ca443ab341ab006801eb8acf232f9eb23087f00687bffed5669c457c59860eb" exitCode=0
Jul 12 18:50:01.850890 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.850717    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"
Jul 12 18:50:01.850890 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.850736    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n" event={"ID":"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8","Type":"ContainerDied","Data":"6ca443ab341ab006801eb8acf232f9eb23087f00687bffed5669c457c59860eb"}
Jul 12 18:50:01.850890 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.850772    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n" event={"ID":"35f778a2-2abb-4bf0-b5af-4226e0aaa5c8","Type":"ContainerDied","Data":"c51f0bbd5585e3403c11c138be4ab9ab820391c61ebbb373b257dabe4dc9645d"}
Jul 12 18:50:01.850890 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.850789    2578 scope.go:117] "RemoveContainer" containerID="6ca443ab341ab006801eb8acf232f9eb23087f00687bffed5669c457c59860eb"
Jul 12 18:50:01.859373 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.859357    2578 scope.go:117] "RemoveContainer" containerID="6ca443ab341ab006801eb8acf232f9eb23087f00687bffed5669c457c59860eb"
Jul 12 18:50:01.859646 ip-10-0-131-122 kubenswrapper[2578]: E0712 18:50:01.859628    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ca443ab341ab006801eb8acf232f9eb23087f00687bffed5669c457c59860eb\": container with ID starting with 6ca443ab341ab006801eb8acf232f9eb23087f00687bffed5669c457c59860eb not found: ID does not exist" containerID="6ca443ab341ab006801eb8acf232f9eb23087f00687bffed5669c457c59860eb"
Jul 12 18:50:01.859721 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.859654    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ca443ab341ab006801eb8acf232f9eb23087f00687bffed5669c457c59860eb"} err="failed to get container status \"6ca443ab341ab006801eb8acf232f9eb23087f00687bffed5669c457c59860eb\": rpc error: code = NotFound desc = could not find container \"6ca443ab341ab006801eb8acf232f9eb23087f00687bffed5669c457c59860eb\": container with ID starting with 6ca443ab341ab006801eb8acf232f9eb23087f00687bffed5669c457c59860eb not found: ID does not exist"
Jul 12 18:50:01.867304 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.867281    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"]
Jul 12 18:50:01.871434 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:01.871404    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-d2f6a5/test-model-d2f6a5-kserve-6bd6b48f-zrf6n"]
Jul 12 18:50:03.326470 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:50:03.326439    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35f778a2-2abb-4bf0-b5af-4226e0aaa5c8" path="/var/lib/kubelet/pods/35f778a2-2abb-4bf0-b5af-4226e0aaa5c8/volumes"
Jul 12 18:53:45.530380 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:53:45.530280    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:53:45.539895 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:53:45.539875    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:58:45.561290 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:58:45.561182    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 18:58:45.571197 ip-10-0-131-122 kubenswrapper[2578]: I0712 18:58:45.571178    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 19:03:25.106230 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.106198    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"]
Jul 12 19:03:25.106662 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.106607    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35f778a2-2abb-4bf0-b5af-4226e0aaa5c8" containerName="main"
Jul 12 19:03:25.106662 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.106619    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="35f778a2-2abb-4bf0-b5af-4226e0aaa5c8" containerName="main"
Jul 12 19:03:25.106662 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.106630    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6bf7ae7-14c7-4792-8580-0a4e46ae3a31" containerName="main"
Jul 12 19:03:25.106662 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.106635    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6bf7ae7-14c7-4792-8580-0a4e46ae3a31" containerName="main"
Jul 12 19:03:25.106811 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.106700    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="35f778a2-2abb-4bf0-b5af-4226e0aaa5c8" containerName="main"
Jul 12 19:03:25.106811 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.106710    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6bf7ae7-14c7-4792-8580-0a4e46ae3a31" containerName="main"
Jul 12 19:03:25.110004 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.109988    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.112591 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.112569    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e6a009\"/\"openshift-service-ca.crt\""
Jul 12 19:03:25.113657 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.113638    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e6a009\"/\"auth-test-model-e6a009-kserve-self-signed-certs\""
Jul 12 19:03:25.113772 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.113695    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e6a009\"/\"default-dockercfg-vgl9x\""
Jul 12 19:03:25.113911 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.113895    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e6a009\"/\"kube-root-ca.crt\""
Jul 12 19:03:25.118401 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.118378    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"]
Jul 12 19:03:25.192070 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.192046    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-home\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.192228 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.192101    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6thgx\" (UniqueName: \"kubernetes.io/projected/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-kube-api-access-6thgx\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.192228 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.192179    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-tls-certs\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.192228 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.192214    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-model-cache\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.192374 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.192260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-tmp-dir\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.192374 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.192290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-dshm\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.292697 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.292674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-dshm\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.292809 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.292709    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-home\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.292809 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.292763    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6thgx\" (UniqueName: \"kubernetes.io/projected/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-kube-api-access-6thgx\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.292809 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.292794    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-tls-certs\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.292978 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.292809    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-model-cache\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.292978 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.292851    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-tmp-dir\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.293182 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.293151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-home\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.293296 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.293201    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-tmp-dir\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.293296 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.293230    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-model-cache\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.294906 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.294883    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-dshm\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.295184 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.295164    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-tls-certs\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.304193 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.304172    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6thgx\" (UniqueName: \"kubernetes.io/projected/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-kube-api-access-6thgx\") pod \"auth-test-model-e6a009-kserve-8c55ddd99-hzlw2\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") " pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.419835 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.419759    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:25.548055 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.547859    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"]
Jul 12 19:03:25.550252 ip-10-0-131-122 kubenswrapper[2578]: W0712 19:03:25.550227    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6279f61e_b7ee_48e3_b3c5_3ecf05ba173d.slice/crio-7cae8c91c91eb2371cc26e5496a52e6e233823ae661b037c580a0b86a2ea4bc7 WatchSource:0}: Error finding container 7cae8c91c91eb2371cc26e5496a52e6e233823ae661b037c580a0b86a2ea4bc7: Status 404 returned error can't find the container with id 7cae8c91c91eb2371cc26e5496a52e6e233823ae661b037c580a0b86a2ea4bc7
Jul 12 19:03:25.737667 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.737633    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2" event={"ID":"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d","Type":"ContainerStarted","Data":"ae7d639acd355aa004497c396e491503763e0c4241473ac4d2c0c37435ff8366"}
Jul 12 19:03:25.737821 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.737679    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2" event={"ID":"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d","Type":"ContainerStarted","Data":"7cae8c91c91eb2371cc26e5496a52e6e233823ae661b037c580a0b86a2ea4bc7"}
Jul 12 19:03:25.753889 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:25.753844    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2" podStartSLOduration=0.753829365 podStartE2EDuration="753.829365ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 19:03:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:03:25.753766078 +0000 UTC m=+4180.953885731" watchObservedRunningTime="2026-07-12 19:03:25.753829365 +0000 UTC m=+4180.953949019"
Jul 12 19:03:35.420008 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:35.419931    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:35.420585 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:35.420032    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:35.432935 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:35.432913    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:35.807386 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:35.807359    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:03:45.591816 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:45.591711    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 19:03:45.603493 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:45.603470    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 19:03:46.196137 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.196106    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"]
Jul 12 19:03:46.203042 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.201652    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.206716 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.206623    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-53dad9\"/\"openshift-service-ca.crt\""
Jul 12 19:03:46.207092 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.207072    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-53dad9\"/\"default-dockercfg-nk6zr\""
Jul 12 19:03:46.207557 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.207536    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-53dad9\"/\"kube-root-ca.crt\""
Jul 12 19:03:46.207665 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.207649    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-53dad9\"/\"auth-test-model-53dad9-kserve-self-signed-certs\""
Jul 12 19:03:46.211574 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.211510    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"]
Jul 12 19:03:46.288507 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.288476    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-dshm\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.288668 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.288510    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92a12d72-ef3f-4181-996e-8f077be3da0d-tls-certs\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.288722 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.288699    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-tmp-dir\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.288770 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.288755    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrs48\" (UniqueName: \"kubernetes.io/projected/92a12d72-ef3f-4181-996e-8f077be3da0d-kube-api-access-rrs48\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.288805 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.288779    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-home\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.288805 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.288797    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-model-cache\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.389745 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.389710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-tmp-dir\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.389886 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.389755    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrs48\" (UniqueName: \"kubernetes.io/projected/92a12d72-ef3f-4181-996e-8f077be3da0d-kube-api-access-rrs48\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.389886 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.389777    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-home\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.389886 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.389794    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-model-cache\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.389886 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.389825    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-dshm\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.389886 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.389845    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92a12d72-ef3f-4181-996e-8f077be3da0d-tls-certs\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.390167 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.390144    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-tmp-dir\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.390167 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.390161    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-home\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.390310 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.390289    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-model-cache\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.392083 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.392059    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-dshm\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.392455 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.392435    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92a12d72-ef3f-4181-996e-8f077be3da0d-tls-certs\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.397489 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.397467    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrs48\" (UniqueName: \"kubernetes.io/projected/92a12d72-ef3f-4181-996e-8f077be3da0d-kube-api-access-rrs48\") pod \"auth-test-model-53dad9-kserve-54cc59bb87-2g5rl\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") " pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.515555 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.515455    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:46.645997 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.645969    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"]
Jul 12 19:03:46.647749 ip-10-0-131-122 kubenswrapper[2578]: W0712 19:03:46.647720    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92a12d72_ef3f_4181_996e_8f077be3da0d.slice/crio-418e31ef5531c2e71e76a8f4914d3422d71709fa6b168986f683c869baffc643 WatchSource:0}: Error finding container 418e31ef5531c2e71e76a8f4914d3422d71709fa6b168986f683c869baffc643: Status 404 returned error can't find the container with id 418e31ef5531c2e71e76a8f4914d3422d71709fa6b168986f683c869baffc643
Jul 12 19:03:46.854576 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.854487    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl" event={"ID":"92a12d72-ef3f-4181-996e-8f077be3da0d","Type":"ContainerStarted","Data":"24ca0d3b0fd0b54e737f0a1f1c9257d237734b255673a5ee968a5e44f093e5e7"}
Jul 12 19:03:46.854576 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.854542    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl" event={"ID":"92a12d72-ef3f-4181-996e-8f077be3da0d","Type":"ContainerStarted","Data":"418e31ef5531c2e71e76a8f4914d3422d71709fa6b168986f683c869baffc643"}
Jul 12 19:03:46.870776 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:46.870730    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl" podStartSLOduration=0.870715843 podStartE2EDuration="870.715843ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 19:03:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:03:46.86942016 +0000 UTC m=+4202.069539820" watchObservedRunningTime="2026-07-12 19:03:46.870715843 +0000 UTC m=+4202.070835498"
Jul 12 19:03:56.515994 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:56.515958    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:56.516390 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:56.516013    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:56.528573 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:56.528541    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:03:56.914650 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:03:56.914568    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:05:50.411750 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:05:50.411666    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"]
Jul 12 19:05:51.200691 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:05:51.200660    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"]
Jul 12 19:06:00.804869 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:00.804830    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"]
Jul 12 19:06:00.808186 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:00.808164    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:00.810393 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:00.810373    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e6a009\"/\"sub-test-model-e6a009-kserve-self-signed-certs\""
Jul 12 19:06:00.814879 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:00.814859    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"]
Jul 12 19:06:00.908223 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:00.908188    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trl2z\" (UniqueName: \"kubernetes.io/projected/9f018748-1e97-439c-a36b-8087a556f4bc-kube-api-access-trl2z\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:00.908374 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:00.908233    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-model-cache\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:00.908374 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:00.908314    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-dshm\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:00.908374 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:00.908340    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-tmp-dir\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:00.908574 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:00.908400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f018748-1e97-439c-a36b-8087a556f4bc-tls-certs\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:00.908574 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:00.908427    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-home\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.009509 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.009481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-dshm\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.009676 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.009537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-tmp-dir\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.009676 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.009583    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f018748-1e97-439c-a36b-8087a556f4bc-tls-certs\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.009676 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.009618    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-home\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.009848 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.009739    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trl2z\" (UniqueName: \"kubernetes.io/projected/9f018748-1e97-439c-a36b-8087a556f4bc-kube-api-access-trl2z\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.009848 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.009780    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-model-cache\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.010018 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.009997    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-tmp-dir\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.010157 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.010064    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-home\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.010157 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.010119    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-model-cache\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.011774 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.011755    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-dshm\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.012094 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.012079    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f018748-1e97-439c-a36b-8087a556f4bc-tls-certs\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.017084 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.017064    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trl2z\" (UniqueName: \"kubernetes.io/projected/9f018748-1e97-439c-a36b-8087a556f4bc-kube-api-access-trl2z\") pod \"sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") " pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.117710 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.117642    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:01.247835 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.247810    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"]
Jul 12 19:06:01.250407 ip-10-0-131-122 kubenswrapper[2578]: W0712 19:06:01.250379    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f018748_1e97_439c_a36b_8087a556f4bc.slice/crio-37d1e267a31d4b6b0e59110de88c9d28ef498291dba3701556df7d8bffe41767 WatchSource:0}: Error finding container 37d1e267a31d4b6b0e59110de88c9d28ef498291dba3701556df7d8bffe41767: Status 404 returned error can't find the container with id 37d1e267a31d4b6b0e59110de88c9d28ef498291dba3701556df7d8bffe41767
Jul 12 19:06:01.516430 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.516396    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2" event={"ID":"9f018748-1e97-439c-a36b-8087a556f4bc","Type":"ContainerStarted","Data":"e7842f1761760e29bf0daa3a13785ee071d539d6a921f88be244b71ef95fe41c"}
Jul 12 19:06:01.516430 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.516436    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2" event={"ID":"9f018748-1e97-439c-a36b-8087a556f4bc","Type":"ContainerStarted","Data":"37d1e267a31d4b6b0e59110de88c9d28ef498291dba3701556df7d8bffe41767"}
Jul 12 19:06:01.532562 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:01.532498    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2" podStartSLOduration=1.532483847 podStartE2EDuration="1.532483847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 19:06:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:06:01.531315239 +0000 UTC m=+4336.731434909" watchObservedRunningTime="2026-07-12 19:06:01.532483847 +0000 UTC m=+4336.732603501"
Jul 12 19:06:05.431672 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.431628    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2" podUID="6279f61e-b7ee-48e3-b3c5-3ecf05ba173d" containerName="main" containerID="cri-o://ae7d639acd355aa004497c396e491503763e0c4241473ac4d2c0c37435ff8366" gracePeriod=45
Jul 12 19:06:05.694460 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.694402    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:06:05.755684 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.755651    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6thgx\" (UniqueName: \"kubernetes.io/projected/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-kube-api-access-6thgx\") pod \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") "
Jul 12 19:06:05.755851 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.755737    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-tls-certs\") pod \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") "
Jul 12 19:06:05.755851 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.755776    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-tmp-dir\") pod \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") "
Jul 12 19:06:05.755851 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.755821    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-dshm\") pod \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") "
Jul 12 19:06:05.756021 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.755887    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-model-cache\") pod \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") "
Jul 12 19:06:05.756021 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.755921    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-home\") pod \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\" (UID: \"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d\") "
Jul 12 19:06:05.756128 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.756037    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6279f61e-b7ee-48e3-b3c5-3ecf05ba173d" (UID: "6279f61e-b7ee-48e3-b3c5-3ecf05ba173d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:06:05.756209 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.756174    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-model-cache" (OuterVolumeSpecName: "model-cache") pod "6279f61e-b7ee-48e3-b3c5-3ecf05ba173d" (UID: "6279f61e-b7ee-48e3-b3c5-3ecf05ba173d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:06:05.756273 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.756235    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-home" (OuterVolumeSpecName: "home") pod "6279f61e-b7ee-48e3-b3c5-3ecf05ba173d" (UID: "6279f61e-b7ee-48e3-b3c5-3ecf05ba173d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:06:05.756333 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.756316    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-tmp-dir\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:06:05.756371 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.756335    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-model-cache\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:06:05.756371 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.756343    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-home\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:06:05.758092 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.758071    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-kube-api-access-6thgx" (OuterVolumeSpecName: "kube-api-access-6thgx") pod "6279f61e-b7ee-48e3-b3c5-3ecf05ba173d" (UID: "6279f61e-b7ee-48e3-b3c5-3ecf05ba173d"). InnerVolumeSpecName "kube-api-access-6thgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 19:06:05.758284 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.758270    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6279f61e-b7ee-48e3-b3c5-3ecf05ba173d" (UID: "6279f61e-b7ee-48e3-b3c5-3ecf05ba173d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 19:06:05.758351 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.758321    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-dshm" (OuterVolumeSpecName: "dshm") pod "6279f61e-b7ee-48e3-b3c5-3ecf05ba173d" (UID: "6279f61e-b7ee-48e3-b3c5-3ecf05ba173d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:06:05.857807 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.857774    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-dshm\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:06:05.857807 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.857801    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6thgx\" (UniqueName: \"kubernetes.io/projected/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-kube-api-access-6thgx\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:06:05.857807 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:05.857811    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d-tls-certs\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:06:06.222046 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.221986    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl" podUID="92a12d72-ef3f-4181-996e-8f077be3da0d" containerName="main" containerID="cri-o://24ca0d3b0fd0b54e737f0a1f1c9257d237734b255673a5ee968a5e44f093e5e7" gracePeriod=45
Jul 12 19:06:06.476291 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.476230    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:06:06.542909 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.542870    2578 generic.go:356] "Generic (PLEG): container finished" podID="6279f61e-b7ee-48e3-b3c5-3ecf05ba173d" containerID="ae7d639acd355aa004497c396e491503763e0c4241473ac4d2c0c37435ff8366" exitCode=0
Jul 12 19:06:06.542909 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.542913    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"
Jul 12 19:06:06.543180 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.542961    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2" event={"ID":"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d","Type":"ContainerDied","Data":"ae7d639acd355aa004497c396e491503763e0c4241473ac4d2c0c37435ff8366"}
Jul 12 19:06:06.543180 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.543006    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2" event={"ID":"6279f61e-b7ee-48e3-b3c5-3ecf05ba173d","Type":"ContainerDied","Data":"7cae8c91c91eb2371cc26e5496a52e6e233823ae661b037c580a0b86a2ea4bc7"}
Jul 12 19:06:06.543180 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.543031    2578 scope.go:117] "RemoveContainer" containerID="ae7d639acd355aa004497c396e491503763e0c4241473ac4d2c0c37435ff8366"
Jul 12 19:06:06.544603 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.544574    2578 generic.go:356] "Generic (PLEG): container finished" podID="92a12d72-ef3f-4181-996e-8f077be3da0d" containerID="24ca0d3b0fd0b54e737f0a1f1c9257d237734b255673a5ee968a5e44f093e5e7" exitCode=0
Jul 12 19:06:06.544733 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.544632    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"
Jul 12 19:06:06.544733 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.544654    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl" event={"ID":"92a12d72-ef3f-4181-996e-8f077be3da0d","Type":"ContainerDied","Data":"24ca0d3b0fd0b54e737f0a1f1c9257d237734b255673a5ee968a5e44f093e5e7"}
Jul 12 19:06:06.544733 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.544702    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl" event={"ID":"92a12d72-ef3f-4181-996e-8f077be3da0d","Type":"ContainerDied","Data":"418e31ef5531c2e71e76a8f4914d3422d71709fa6b168986f683c869baffc643"}
Jul 12 19:06:06.552765 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.552743    2578 scope.go:117] "RemoveContainer" containerID="ae7d639acd355aa004497c396e491503763e0c4241473ac4d2c0c37435ff8366"
Jul 12 19:06:06.553007 ip-10-0-131-122 kubenswrapper[2578]: E0712 19:06:06.552991    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae7d639acd355aa004497c396e491503763e0c4241473ac4d2c0c37435ff8366\": container with ID starting with ae7d639acd355aa004497c396e491503763e0c4241473ac4d2c0c37435ff8366 not found: ID does not exist" containerID="ae7d639acd355aa004497c396e491503763e0c4241473ac4d2c0c37435ff8366"
Jul 12 19:06:06.553078 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.553014    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae7d639acd355aa004497c396e491503763e0c4241473ac4d2c0c37435ff8366"} err="failed to get container status \"ae7d639acd355aa004497c396e491503763e0c4241473ac4d2c0c37435ff8366\": rpc error: code = NotFound desc = could not find container \"ae7d639acd355aa004497c396e491503763e0c4241473ac4d2c0c37435ff8366\": container with ID starting with ae7d639acd355aa004497c396e491503763e0c4241473ac4d2c0c37435ff8366 not found: ID does not exist"
Jul 12 19:06:06.553078 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.553030    2578 scope.go:117] "RemoveContainer" containerID="24ca0d3b0fd0b54e737f0a1f1c9257d237734b255673a5ee968a5e44f093e5e7"
Jul 12 19:06:06.560713 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.560694    2578 scope.go:117] "RemoveContainer" containerID="24ca0d3b0fd0b54e737f0a1f1c9257d237734b255673a5ee968a5e44f093e5e7"
Jul 12 19:06:06.561019 ip-10-0-131-122 kubenswrapper[2578]: E0712 19:06:06.560999    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24ca0d3b0fd0b54e737f0a1f1c9257d237734b255673a5ee968a5e44f093e5e7\": container with ID starting with 24ca0d3b0fd0b54e737f0a1f1c9257d237734b255673a5ee968a5e44f093e5e7 not found: ID does not exist" containerID="24ca0d3b0fd0b54e737f0a1f1c9257d237734b255673a5ee968a5e44f093e5e7"
Jul 12 19:06:06.561114 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.561025    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24ca0d3b0fd0b54e737f0a1f1c9257d237734b255673a5ee968a5e44f093e5e7"} err="failed to get container status \"24ca0d3b0fd0b54e737f0a1f1c9257d237734b255673a5ee968a5e44f093e5e7\": rpc error: code = NotFound desc = could not find container \"24ca0d3b0fd0b54e737f0a1f1c9257d237734b255673a5ee968a5e44f093e5e7\": container with ID starting with 24ca0d3b0fd0b54e737f0a1f1c9257d237734b255673a5ee968a5e44f093e5e7 not found: ID does not exist"
Jul 12 19:06:06.563169 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.563149    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-tmp-dir\") pod \"92a12d72-ef3f-4181-996e-8f077be3da0d\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") "
Jul 12 19:06:06.563274 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.563188    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92a12d72-ef3f-4181-996e-8f077be3da0d-tls-certs\") pod \"92a12d72-ef3f-4181-996e-8f077be3da0d\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") "
Jul 12 19:06:06.563274 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.563236    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-dshm\") pod \"92a12d72-ef3f-4181-996e-8f077be3da0d\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") "
Jul 12 19:06:06.563386 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.563308    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-model-cache\") pod \"92a12d72-ef3f-4181-996e-8f077be3da0d\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") "
Jul 12 19:06:06.563438 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.563380    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "92a12d72-ef3f-4181-996e-8f077be3da0d" (UID: "92a12d72-ef3f-4181-996e-8f077be3da0d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:06:06.563438 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.563424    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrs48\" (UniqueName: \"kubernetes.io/projected/92a12d72-ef3f-4181-996e-8f077be3da0d-kube-api-access-rrs48\") pod \"92a12d72-ef3f-4181-996e-8f077be3da0d\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") "
Jul 12 19:06:06.563621 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.563556    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-home\") pod \"92a12d72-ef3f-4181-996e-8f077be3da0d\" (UID: \"92a12d72-ef3f-4181-996e-8f077be3da0d\") "
Jul 12 19:06:06.563621 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.563578    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-model-cache" (OuterVolumeSpecName: "model-cache") pod "92a12d72-ef3f-4181-996e-8f077be3da0d" (UID: "92a12d72-ef3f-4181-996e-8f077be3da0d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:06:06.563786 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.563765    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"]
Jul 12 19:06:06.563859 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.563840    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-home" (OuterVolumeSpecName: "home") pod "92a12d72-ef3f-4181-996e-8f077be3da0d" (UID: "92a12d72-ef3f-4181-996e-8f077be3da0d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:06:06.563963 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.563945    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-home\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:06:06.564047 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.563966    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-tmp-dir\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:06:06.564047 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.563977    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-model-cache\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:06:06.566032 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.566008    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92a12d72-ef3f-4181-996e-8f077be3da0d-kube-api-access-rrs48" (OuterVolumeSpecName: "kube-api-access-rrs48") pod "92a12d72-ef3f-4181-996e-8f077be3da0d" (UID: "92a12d72-ef3f-4181-996e-8f077be3da0d"). InnerVolumeSpecName "kube-api-access-rrs48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 19:06:06.566157 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.566135    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92a12d72-ef3f-4181-996e-8f077be3da0d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "92a12d72-ef3f-4181-996e-8f077be3da0d" (UID: "92a12d72-ef3f-4181-996e-8f077be3da0d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 19:06:06.566199 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.566143    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-dshm" (OuterVolumeSpecName: "dshm") pod "92a12d72-ef3f-4181-996e-8f077be3da0d" (UID: "92a12d72-ef3f-4181-996e-8f077be3da0d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:06:06.567285 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.567268    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/auth-test-model-e6a009-kserve-8c55ddd99-hzlw2"]
Jul 12 19:06:06.664948 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.664907    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92a12d72-ef3f-4181-996e-8f077be3da0d-tls-certs\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:06:06.664948 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.664942    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92a12d72-ef3f-4181-996e-8f077be3da0d-dshm\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:06:06.664948 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.664951    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrs48\" (UniqueName: \"kubernetes.io/projected/92a12d72-ef3f-4181-996e-8f077be3da0d-kube-api-access-rrs48\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:06:06.871652 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.871619    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"]
Jul 12 19:06:06.876007 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:06.875976    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/auth-test-model-53dad9-kserve-54cc59bb87-2g5rl"]
Jul 12 19:06:07.326485 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:07.326439    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6279f61e-b7ee-48e3-b3c5-3ecf05ba173d" path="/var/lib/kubelet/pods/6279f61e-b7ee-48e3-b3c5-3ecf05ba173d/volumes"
Jul 12 19:06:07.326972 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:07.326950    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92a12d72-ef3f-4181-996e-8f077be3da0d" path="/var/lib/kubelet/pods/92a12d72-ef3f-4181-996e-8f077be3da0d/volumes"
Jul 12 19:06:11.118607 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:11.118575    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:11.118975 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:11.118620    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:11.131055 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:11.131025    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:11.582977 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:11.582951    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:06:32.906622 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.906585    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"]
Jul 12 19:06:32.907263 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.907142    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92a12d72-ef3f-4181-996e-8f077be3da0d" containerName="main"
Jul 12 19:06:32.907263 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.907162    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="92a12d72-ef3f-4181-996e-8f077be3da0d" containerName="main"
Jul 12 19:06:32.907263 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.907229    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6279f61e-b7ee-48e3-b3c5-3ecf05ba173d" containerName="main"
Jul 12 19:06:32.907263 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.907240    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6279f61e-b7ee-48e3-b3c5-3ecf05ba173d" containerName="main"
Jul 12 19:06:32.907495 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.907364    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="92a12d72-ef3f-4181-996e-8f077be3da0d" containerName="main"
Jul 12 19:06:32.907495 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.907387    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6279f61e-b7ee-48e3-b3c5-3ecf05ba173d" containerName="main"
Jul 12 19:06:32.910288 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.910268    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:32.913812 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.913792    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-53dad9\"/\"sub-test-model-53dad9-kserve-self-signed-certs\""
Jul 12 19:06:32.913918 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.913809    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-53dad9\"/\"default-dockercfg-nk6zr\""
Jul 12 19:06:32.913918 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.913816    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-53dad9\"/\"openshift-service-ca.crt\""
Jul 12 19:06:32.913918 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.913855    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-53dad9\"/\"kube-root-ca.crt\""
Jul 12 19:06:32.918547 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.918503    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"]
Jul 12 19:06:32.996068 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.996039    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-dshm\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:32.996184 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.996071    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-tmp-dir\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:32.996184 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.996100    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8jl7\" (UniqueName: \"kubernetes.io/projected/7b7bfc94-5a3d-4914-b83e-5b99a9599218-kube-api-access-h8jl7\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:32.996184 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.996160    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-model-cache\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:32.996305 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.996186    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b7bfc94-5a3d-4914-b83e-5b99a9599218-tls-certs\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:32.996305 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:32.996220    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-home\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.098263 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.097117    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8jl7\" (UniqueName: \"kubernetes.io/projected/7b7bfc94-5a3d-4914-b83e-5b99a9599218-kube-api-access-h8jl7\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.098263 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.097205    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-model-cache\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.098263 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.097248    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b7bfc94-5a3d-4914-b83e-5b99a9599218-tls-certs\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.098263 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.097304    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-home\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.098263 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.097347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-dshm\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.098263 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.097385    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-tmp-dir\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.098263 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.097875    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-tmp-dir\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.098819 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.098271    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-home\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.098819 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.098421    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-model-cache\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.100427 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.100406    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-dshm\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.100869 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.100851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b7bfc94-5a3d-4914-b83e-5b99a9599218-tls-certs\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.105325 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.105302    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8jl7\" (UniqueName: \"kubernetes.io/projected/7b7bfc94-5a3d-4914-b83e-5b99a9599218-kube-api-access-h8jl7\") pod \"sub-test-model-53dad9-kserve-849d5696dc-5c5rg\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") " pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.221145 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.221105    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:33.367175 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.367152    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"]
Jul 12 19:06:33.369025 ip-10-0-131-122 kubenswrapper[2578]: W0712 19:06:33.369000    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b7bfc94_5a3d_4914_b83e_5b99a9599218.slice/crio-b6277acca4336e0dd5b5ae409870e0819f3469f2c9aec574d4ce6e900ad9705d WatchSource:0}: Error finding container b6277acca4336e0dd5b5ae409870e0819f3469f2c9aec574d4ce6e900ad9705d: Status 404 returned error can't find the container with id b6277acca4336e0dd5b5ae409870e0819f3469f2c9aec574d4ce6e900ad9705d
Jul 12 19:06:33.694800 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.694769    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg" event={"ID":"7b7bfc94-5a3d-4914-b83e-5b99a9599218","Type":"ContainerStarted","Data":"0986225cddcb6204518c8169ef34f9a85752e3dd6ceabb44ec4ae060a231de36"}
Jul 12 19:06:33.694800 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.694806    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg" event={"ID":"7b7bfc94-5a3d-4914-b83e-5b99a9599218","Type":"ContainerStarted","Data":"b6277acca4336e0dd5b5ae409870e0819f3469f2c9aec574d4ce6e900ad9705d"}
Jul 12 19:06:33.711557 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:33.711499    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg" podStartSLOduration=1.7114859500000001 podStartE2EDuration="1.71148595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 19:06:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:06:33.709695796 +0000 UTC m=+4368.909815449" watchObservedRunningTime="2026-07-12 19:06:33.71148595 +0000 UTC m=+4368.911605603"
Jul 12 19:06:43.221448 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:43.221405    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:43.221448 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:43.221456    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:43.235755 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:43.235723    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:06:43.755817 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:06:43.755784    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:07:13.852648 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:13.852561    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"]
Jul 12 19:07:14.786167 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:14.786134    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"]
Jul 12 19:07:28.873581 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:28.873503    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2" podUID="9f018748-1e97-439c-a36b-8087a556f4bc" containerName="main" containerID="cri-o://e7842f1761760e29bf0daa3a13785ee071d539d6a921f88be244b71ef95fe41c" gracePeriod=45
Jul 12 19:07:29.123638 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.123580    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:07:29.142031 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.141993    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-dshm\") pod \"9f018748-1e97-439c-a36b-8087a556f4bc\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") "
Jul 12 19:07:29.142031 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.142030    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-model-cache\") pod \"9f018748-1e97-439c-a36b-8087a556f4bc\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") "
Jul 12 19:07:29.142259 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.142063    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-home\") pod \"9f018748-1e97-439c-a36b-8087a556f4bc\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") "
Jul 12 19:07:29.142259 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.142157    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f018748-1e97-439c-a36b-8087a556f4bc-tls-certs\") pod \"9f018748-1e97-439c-a36b-8087a556f4bc\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") "
Jul 12 19:07:29.142259 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.142223    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trl2z\" (UniqueName: \"kubernetes.io/projected/9f018748-1e97-439c-a36b-8087a556f4bc-kube-api-access-trl2z\") pod \"9f018748-1e97-439c-a36b-8087a556f4bc\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") "
Jul 12 19:07:29.142424 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.142290    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-tmp-dir\") pod \"9f018748-1e97-439c-a36b-8087a556f4bc\" (UID: \"9f018748-1e97-439c-a36b-8087a556f4bc\") "
Jul 12 19:07:29.142424 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.142329    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-model-cache" (OuterVolumeSpecName: "model-cache") pod "9f018748-1e97-439c-a36b-8087a556f4bc" (UID: "9f018748-1e97-439c-a36b-8087a556f4bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:07:29.142424 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.142363    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-home" (OuterVolumeSpecName: "home") pod "9f018748-1e97-439c-a36b-8087a556f4bc" (UID: "9f018748-1e97-439c-a36b-8087a556f4bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:07:29.142684 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.142662    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-model-cache\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:07:29.142800 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.142686    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-home\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:07:29.143354 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.143147    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9f018748-1e97-439c-a36b-8087a556f4bc" (UID: "9f018748-1e97-439c-a36b-8087a556f4bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:07:29.144677 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.144644    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-dshm" (OuterVolumeSpecName: "dshm") pod "9f018748-1e97-439c-a36b-8087a556f4bc" (UID: "9f018748-1e97-439c-a36b-8087a556f4bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:07:29.144784 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.144740    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f018748-1e97-439c-a36b-8087a556f4bc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9f018748-1e97-439c-a36b-8087a556f4bc" (UID: "9f018748-1e97-439c-a36b-8087a556f4bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 19:07:29.144950 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.144926    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f018748-1e97-439c-a36b-8087a556f4bc-kube-api-access-trl2z" (OuterVolumeSpecName: "kube-api-access-trl2z") pod "9f018748-1e97-439c-a36b-8087a556f4bc" (UID: "9f018748-1e97-439c-a36b-8087a556f4bc"). InnerVolumeSpecName "kube-api-access-trl2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 19:07:29.243689 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.243644    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f018748-1e97-439c-a36b-8087a556f4bc-tls-certs\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:07:29.243689 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.243684    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trl2z\" (UniqueName: \"kubernetes.io/projected/9f018748-1e97-439c-a36b-8087a556f4bc-kube-api-access-trl2z\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:07:29.243689 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.243696    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-tmp-dir\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:07:29.243689 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.243705    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f018748-1e97-439c-a36b-8087a556f4bc-dshm\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:07:29.806394 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.806330    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg" podUID="7b7bfc94-5a3d-4914-b83e-5b99a9599218" containerName="main" containerID="cri-o://0986225cddcb6204518c8169ef34f9a85752e3dd6ceabb44ec4ae060a231de36" gracePeriod=45
Jul 12 19:07:29.968544 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.968500    2578 generic.go:356] "Generic (PLEG): container finished" podID="9f018748-1e97-439c-a36b-8087a556f4bc" containerID="e7842f1761760e29bf0daa3a13785ee071d539d6a921f88be244b71ef95fe41c" exitCode=0
Jul 12 19:07:29.968991 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.968548    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2" event={"ID":"9f018748-1e97-439c-a36b-8087a556f4bc","Type":"ContainerDied","Data":"e7842f1761760e29bf0daa3a13785ee071d539d6a921f88be244b71ef95fe41c"}
Jul 12 19:07:29.968991 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.968567    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"
Jul 12 19:07:29.968991 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.968595    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2" event={"ID":"9f018748-1e97-439c-a36b-8087a556f4bc","Type":"ContainerDied","Data":"37d1e267a31d4b6b0e59110de88c9d28ef498291dba3701556df7d8bffe41767"}
Jul 12 19:07:29.968991 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.968618    2578 scope.go:117] "RemoveContainer" containerID="e7842f1761760e29bf0daa3a13785ee071d539d6a921f88be244b71ef95fe41c"
Jul 12 19:07:29.970721 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.970696    2578 generic.go:356] "Generic (PLEG): container finished" podID="7b7bfc94-5a3d-4914-b83e-5b99a9599218" containerID="0986225cddcb6204518c8169ef34f9a85752e3dd6ceabb44ec4ae060a231de36" exitCode=0
Jul 12 19:07:29.970837 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.970817    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg" event={"ID":"7b7bfc94-5a3d-4914-b83e-5b99a9599218","Type":"ContainerDied","Data":"0986225cddcb6204518c8169ef34f9a85752e3dd6ceabb44ec4ae060a231de36"}
Jul 12 19:07:29.978089 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.978026    2578 scope.go:117] "RemoveContainer" containerID="e7842f1761760e29bf0daa3a13785ee071d539d6a921f88be244b71ef95fe41c"
Jul 12 19:07:29.978315 ip-10-0-131-122 kubenswrapper[2578]: E0712 19:07:29.978290    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7842f1761760e29bf0daa3a13785ee071d539d6a921f88be244b71ef95fe41c\": container with ID starting with e7842f1761760e29bf0daa3a13785ee071d539d6a921f88be244b71ef95fe41c not found: ID does not exist" containerID="e7842f1761760e29bf0daa3a13785ee071d539d6a921f88be244b71ef95fe41c"
Jul 12 19:07:29.978401 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.978323    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7842f1761760e29bf0daa3a13785ee071d539d6a921f88be244b71ef95fe41c"} err="failed to get container status \"e7842f1761760e29bf0daa3a13785ee071d539d6a921f88be244b71ef95fe41c\": rpc error: code = NotFound desc = could not find container \"e7842f1761760e29bf0daa3a13785ee071d539d6a921f88be244b71ef95fe41c\": container with ID starting with e7842f1761760e29bf0daa3a13785ee071d539d6a921f88be244b71ef95fe41c not found: ID does not exist"
Jul 12 19:07:29.985892 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.985866    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"]
Jul 12 19:07:29.991501 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:29.991217    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/sub-test-model-e6a009-kserve-7dbfcd76c7-8qxp2"]
Jul 12 19:07:30.061955 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.061902    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:07:30.152493 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.152459    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-home\") pod \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") "
Jul 12 19:07:30.152714 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.152564    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-dshm\") pod \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") "
Jul 12 19:07:30.152714 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.152612    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-model-cache\") pod \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") "
Jul 12 19:07:30.152714 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.152666    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-tmp-dir\") pod \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") "
Jul 12 19:07:30.152714 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.152708    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8jl7\" (UniqueName: \"kubernetes.io/projected/7b7bfc94-5a3d-4914-b83e-5b99a9599218-kube-api-access-h8jl7\") pod \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") "
Jul 12 19:07:30.152937 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.152731    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b7bfc94-5a3d-4914-b83e-5b99a9599218-tls-certs\") pod \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\" (UID: \"7b7bfc94-5a3d-4914-b83e-5b99a9599218\") "
Jul 12 19:07:30.152937 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.152824    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-home" (OuterVolumeSpecName: "home") pod "7b7bfc94-5a3d-4914-b83e-5b99a9599218" (UID: "7b7bfc94-5a3d-4914-b83e-5b99a9599218"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:07:30.152937 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.152881    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7b7bfc94-5a3d-4914-b83e-5b99a9599218" (UID: "7b7bfc94-5a3d-4914-b83e-5b99a9599218"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:07:30.152937 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.152913    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-model-cache" (OuterVolumeSpecName: "model-cache") pod "7b7bfc94-5a3d-4914-b83e-5b99a9599218" (UID: "7b7bfc94-5a3d-4914-b83e-5b99a9599218"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:07:30.153149 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.153099    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-tmp-dir\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:07:30.153149 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.153117    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-home\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:07:30.153149 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.153126    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-model-cache\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:07:30.154770 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.154741    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-dshm" (OuterVolumeSpecName: "dshm") pod "7b7bfc94-5a3d-4914-b83e-5b99a9599218" (UID: "7b7bfc94-5a3d-4914-b83e-5b99a9599218"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:07:30.154883 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.154795    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b7bfc94-5a3d-4914-b83e-5b99a9599218-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7b7bfc94-5a3d-4914-b83e-5b99a9599218" (UID: "7b7bfc94-5a3d-4914-b83e-5b99a9599218"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 19:07:30.154883 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.154830    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b7bfc94-5a3d-4914-b83e-5b99a9599218-kube-api-access-h8jl7" (OuterVolumeSpecName: "kube-api-access-h8jl7") pod "7b7bfc94-5a3d-4914-b83e-5b99a9599218" (UID: "7b7bfc94-5a3d-4914-b83e-5b99a9599218"). InnerVolumeSpecName "kube-api-access-h8jl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 19:07:30.253697 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.253660    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8jl7\" (UniqueName: \"kubernetes.io/projected/7b7bfc94-5a3d-4914-b83e-5b99a9599218-kube-api-access-h8jl7\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:07:30.253697 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.253688    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b7bfc94-5a3d-4914-b83e-5b99a9599218-tls-certs\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:07:30.253697 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.253697    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b7bfc94-5a3d-4914-b83e-5b99a9599218-dshm\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:07:30.977418 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.977376    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg" event={"ID":"7b7bfc94-5a3d-4914-b83e-5b99a9599218","Type":"ContainerDied","Data":"b6277acca4336e0dd5b5ae409870e0819f3469f2c9aec574d4ce6e900ad9705d"}
Jul 12 19:07:30.977852 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.977433    2578 scope.go:117] "RemoveContainer" containerID="0986225cddcb6204518c8169ef34f9a85752e3dd6ceabb44ec4ae060a231de36"
Jul 12 19:07:30.977852 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.977388    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"
Jul 12 19:07:30.998495 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:30.998467    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"]
Jul 12 19:07:31.001328 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:31.001301    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/sub-test-model-53dad9-kserve-849d5696dc-5c5rg"]
Jul 12 19:07:31.326467 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:31.326380    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b7bfc94-5a3d-4914-b83e-5b99a9599218" path="/var/lib/kubelet/pods/7b7bfc94-5a3d-4914-b83e-5b99a9599218/volumes"
Jul 12 19:07:31.326873 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:31.326857    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f018748-1e97-439c-a36b-8087a556f4bc" path="/var/lib/kubelet/pods/9f018748-1e97-439c-a36b-8087a556f4bc/volumes"
Jul 12 19:07:51.904671 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:51.904628    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"]
Jul 12 19:07:51.905249 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:51.905230    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b7bfc94-5a3d-4914-b83e-5b99a9599218" containerName="main"
Jul 12 19:07:51.905249 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:51.905250    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b7bfc94-5a3d-4914-b83e-5b99a9599218" containerName="main"
Jul 12 19:07:51.905389 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:51.905272    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f018748-1e97-439c-a36b-8087a556f4bc" containerName="main"
Jul 12 19:07:51.905389 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:51.905282    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f018748-1e97-439c-a36b-8087a556f4bc" containerName="main"
Jul 12 19:07:51.905499 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:51.905389    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b7bfc94-5a3d-4914-b83e-5b99a9599218" containerName="main"
Jul 12 19:07:51.905499 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:51.905403    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f018748-1e97-439c-a36b-8087a556f4bc" containerName="main"
Jul 12 19:07:51.910313 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:51.910294    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:51.914042 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:51.914015    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-53dad9\"/\"rate-test-model-53dad9-kserve-self-signed-certs\""
Jul 12 19:07:51.914761 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:51.914020    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-53dad9\"/\"openshift-service-ca.crt\""
Jul 12 19:07:51.914878 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:51.914142    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-53dad9\"/\"kube-root-ca.crt\""
Jul 12 19:07:51.914945 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:51.914198    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-53dad9\"/\"default-dockercfg-nk6zr\""
Jul 12 19:07:51.916862 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:51.916825    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"]
Jul 12 19:07:52.066697 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.066652    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-dshm\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.066697 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.066703    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-home\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.066910 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.066755    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-tmp-dir\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.066910 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.066812    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-model-cache\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.066910 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.066832    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c892q\" (UniqueName: \"kubernetes.io/projected/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-kube-api-access-c892q\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.066910 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.066853    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-tls-certs\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.167668 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.167573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-home\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.167668 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.167616    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-tmp-dir\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.167668 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.167664    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-model-cache\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.167953 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.167691    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c892q\" (UniqueName: \"kubernetes.io/projected/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-kube-api-access-c892q\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.167953 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.167721    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-tls-certs\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.167953 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.167834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-dshm\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.168077 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.168028    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-home\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.168077 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.168043    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-model-cache\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.168150 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.168135    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-tmp-dir\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.169990 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.169962    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-dshm\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.170320 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.170299    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-tls-certs\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.176163 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.176137    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c892q\" (UniqueName: \"kubernetes.io/projected/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-kube-api-access-c892q\") pod \"rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") " pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.221528 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.221492    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:07:52.356013 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:52.355987    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"]
Jul 12 19:07:52.357935 ip-10-0-131-122 kubenswrapper[2578]: W0712 19:07:52.357905    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0e49858_7bf1_42a9_a141_35c41ea6f3c8.slice/crio-fb29315d6743e95af9a55fd058a5d3656478e7084f85426b61e510275d57a57f WatchSource:0}: Error finding container fb29315d6743e95af9a55fd058a5d3656478e7084f85426b61e510275d57a57f: Status 404 returned error can't find the container with id fb29315d6743e95af9a55fd058a5d3656478e7084f85426b61e510275d57a57f
Jul 12 19:07:53.094465 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:53.094431    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb" event={"ID":"d0e49858-7bf1-42a9-a141-35c41ea6f3c8","Type":"ContainerStarted","Data":"74b857071f698ec45d7194fc7850fbd14d0eed87ad0944b27fd66fc65ae04907"}
Jul 12 19:07:53.094865 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:53.094473    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb" event={"ID":"d0e49858-7bf1-42a9-a141-35c41ea6f3c8","Type":"ContainerStarted","Data":"fb29315d6743e95af9a55fd058a5d3656478e7084f85426b61e510275d57a57f"}
Jul 12 19:07:53.111666 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:07:53.111622    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb" podStartSLOduration=2.111607406 podStartE2EDuration="2.111607406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 19:07:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:07:53.110201616 +0000 UTC m=+4448.310321284" watchObservedRunningTime="2026-07-12 19:07:53.111607406 +0000 UTC m=+4448.311727060"
Jul 12 19:08:02.222032 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:02.221991    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:08:02.222472 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:02.222054    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:08:02.234585 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:02.234558    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:08:03.153402 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:03.153375    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:08:30.108132 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:30.108094    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"]
Jul 12 19:08:45.128911 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.128808    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb" podUID="d0e49858-7bf1-42a9-a141-35c41ea6f3c8" containerName="main" containerID="cri-o://74b857071f698ec45d7194fc7850fbd14d0eed87ad0944b27fd66fc65ae04907" gracePeriod=45
Jul 12 19:08:45.356852 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.356815    2578 generic.go:356] "Generic (PLEG): container finished" podID="d0e49858-7bf1-42a9-a141-35c41ea6f3c8" containerID="74b857071f698ec45d7194fc7850fbd14d0eed87ad0944b27fd66fc65ae04907" exitCode=0
Jul 12 19:08:45.357022 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.356867    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb" event={"ID":"d0e49858-7bf1-42a9-a141-35c41ea6f3c8","Type":"ContainerDied","Data":"74b857071f698ec45d7194fc7850fbd14d0eed87ad0944b27fd66fc65ae04907"}
Jul 12 19:08:45.382997 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.382934    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:08:45.475194 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.475158    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-dshm\") pod \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") "
Jul 12 19:08:45.475194 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.475197    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-tls-certs\") pod \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") "
Jul 12 19:08:45.475452 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.475225    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c892q\" (UniqueName: \"kubernetes.io/projected/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-kube-api-access-c892q\") pod \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") "
Jul 12 19:08:45.475452 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.475335    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-model-cache\") pod \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") "
Jul 12 19:08:45.475600 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.475463    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-home\") pod \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") "
Jul 12 19:08:45.475600 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.475492    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-tmp-dir\") pod \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\" (UID: \"d0e49858-7bf1-42a9-a141-35c41ea6f3c8\") "
Jul 12 19:08:45.475695 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.475632    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0e49858-7bf1-42a9-a141-35c41ea6f3c8" (UID: "d0e49858-7bf1-42a9-a141-35c41ea6f3c8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:08:45.475732 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.475689    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-home" (OuterVolumeSpecName: "home") pod "d0e49858-7bf1-42a9-a141-35c41ea6f3c8" (UID: "d0e49858-7bf1-42a9-a141-35c41ea6f3c8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:08:45.475840 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.475817    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0e49858-7bf1-42a9-a141-35c41ea6f3c8" (UID: "d0e49858-7bf1-42a9-a141-35c41ea6f3c8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:08:45.475840 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.475833    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-model-cache\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:08:45.475907 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.475847    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-home\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:08:45.477388 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.477357    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-dshm" (OuterVolumeSpecName: "dshm") pod "d0e49858-7bf1-42a9-a141-35c41ea6f3c8" (UID: "d0e49858-7bf1-42a9-a141-35c41ea6f3c8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 19:08:45.477512 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.477399    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0e49858-7bf1-42a9-a141-35c41ea6f3c8" (UID: "d0e49858-7bf1-42a9-a141-35c41ea6f3c8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 19:08:45.477512 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.477471    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-kube-api-access-c892q" (OuterVolumeSpecName: "kube-api-access-c892q") pod "d0e49858-7bf1-42a9-a141-35c41ea6f3c8" (UID: "d0e49858-7bf1-42a9-a141-35c41ea6f3c8"). InnerVolumeSpecName "kube-api-access-c892q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 19:08:45.577097 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.577064    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-tmp-dir\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:08:45.577097 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.577088    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-dshm\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:08:45.577097 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.577097    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-tls-certs\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:08:45.577097 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.577106    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c892q\" (UniqueName: \"kubernetes.io/projected/d0e49858-7bf1-42a9-a141-35c41ea6f3c8-kube-api-access-c892q\") on node \"ip-10-0-131-122.ec2.internal\" DevicePath \"\""
Jul 12 19:08:45.622720 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.622612    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 19:08:45.635803 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:45.635730    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 19:08:46.363247 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:46.363214    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"
Jul 12 19:08:46.363247 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:46.363232    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb" event={"ID":"d0e49858-7bf1-42a9-a141-35c41ea6f3c8","Type":"ContainerDied","Data":"fb29315d6743e95af9a55fd058a5d3656478e7084f85426b61e510275d57a57f"}
Jul 12 19:08:46.363799 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:46.363294    2578 scope.go:117] "RemoveContainer" containerID="74b857071f698ec45d7194fc7850fbd14d0eed87ad0944b27fd66fc65ae04907"
Jul 12 19:08:46.383458 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:46.383431    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"]
Jul 12 19:08:46.387403 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:46.387377    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-53dad9/rate-test-model-53dad9-kserve-d5cdcb5d7-qv6qb"]
Jul 12 19:08:47.326155 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:08:47.326112    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0e49858-7bf1-42a9-a141-35c41ea6f3c8" path="/var/lib/kubelet/pods/d0e49858-7bf1-42a9-a141-35c41ea6f3c8/volumes"
Jul 12 19:13:45.653464 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:13:45.653352    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 19:13:45.667352 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:13:45.667330    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-trbvl_d02b4365-fc9c-4ce5-ad3c-15b74ec8cd08/ovn-acl-logging/0.log"
Jul 12 19:15:00.399991 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:00.399951    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tpvlh_206a48fd-5615-432e-8d3c-f23279b775e9/manager/0.log"
Jul 12 19:15:00.512031 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:00.512006    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5dc7598fc6-txzwt_c8a6658f-4dfa-4c54-8f78-8155674e1e58/manager/0.log"
Jul 12 19:15:00.974416 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:00.974387    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv_2bd5a084-566d-436d-a7af-d65fd1c64ceb/copy-manifests/0.log"
Jul 12 19:15:00.980167 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:00.980145    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv_2bd5a084-566d-436d-a7af-d65fd1c64ceb/manager/0.log"
Jul 12 19:15:03.184414 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:03.184390    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xxnzw_6a08f283-ffe2-4a00-9d5a-4b3c28a483cf/kuadrant-console-plugin/0.log"
Jul 12 19:15:04.329025 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:04.328998    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-795d67df6d-rtl29_50471198-3766-43f5-acf6-f7f859a103fd/kube-auth-proxy/0.log"
Jul 12 19:15:04.680640 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:04.680609    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-tprz5_f7731e1d-da1f-4341-b6ea-436df7790893/payload-processing/0.log"
Jul 12 19:15:05.124247 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:05.124165    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-44nd4_33165e26-93eb-423b-892b-88d630bbf501/main/0.log"
Jul 12 19:15:05.460108 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:05.460080    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtsj7_e4494d08-aae7-4205-a9a0-baf0e32de15f/main/0.log"
Jul 12 19:15:05.682199 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:05.682173    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-686nd_dbdeef30-9eb8-4a86-8ae7-5e5c9d07decc/main/0.log"
Jul 12 19:15:12.683973 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:12.683939    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-97vpp_9d46ded0-0449-4d25-81bd-d3e7c5ae3a40/global-pull-secret-syncer/0.log"
Jul 12 19:15:12.910116 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:12.910087    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jrpgp_61e71b08-89e9-4f55-a6db-a5f0dcd10f2a/konnectivity-agent/0.log"
Jul 12 19:15:12.955735 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:12.955669    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-122.ec2.internal_9235ba932a6066a0b11ef970f30044e4/haproxy/0.log"
Jul 12 19:15:18.293445 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:18.293420    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xxnzw_6a08f283-ffe2-4a00-9d5a-4b3c28a483cf/kuadrant-console-plugin/0.log"
Jul 12 19:15:20.336250 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.336183    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7eab1750-c342-4045-a543-06e6cc42e3e4/alertmanager/0.log"
Jul 12 19:15:20.357528 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.357502    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7eab1750-c342-4045-a543-06e6cc42e3e4/config-reloader/0.log"
Jul 12 19:15:20.382022 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.382004    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7eab1750-c342-4045-a543-06e6cc42e3e4/kube-rbac-proxy-web/0.log"
Jul 12 19:15:20.402909 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.402890    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7eab1750-c342-4045-a543-06e6cc42e3e4/kube-rbac-proxy/0.log"
Jul 12 19:15:20.427089 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.427065    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7eab1750-c342-4045-a543-06e6cc42e3e4/kube-rbac-proxy-metric/0.log"
Jul 12 19:15:20.452392 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.452370    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7eab1750-c342-4045-a543-06e6cc42e3e4/prom-label-proxy/0.log"
Jul 12 19:15:20.473461 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.473444    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7eab1750-c342-4045-a543-06e6cc42e3e4/init-config-reloader/0.log"
Jul 12 19:15:20.584438 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.584417    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-b2nhm_59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd/kube-state-metrics/0.log"
Jul 12 19:15:20.607098 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.607048    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-b2nhm_59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd/kube-rbac-proxy-main/0.log"
Jul 12 19:15:20.629208 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.629189    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-b2nhm_59e19bd9-7fdb-49d9-b2f9-ff446e4c4dcd/kube-rbac-proxy-self/0.log"
Jul 12 19:15:20.653413 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.653396    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6b65dd6d44-rcs4v_4140ff72-996c-40d9-b2f6-40e241f6894e/metrics-server/0.log"
Jul 12 19:15:20.682426 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.682411    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-tp7vq_3b6d4c90-9b6b-4773-b866-88492d05373b/monitoring-plugin/0.log"
Jul 12 19:15:20.787359 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.787342    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7wtg8_f7e4dbf6-2a10-4a43-8bac-d745892049dd/node-exporter/0.log"
Jul 12 19:15:20.806399 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.806384    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7wtg8_f7e4dbf6-2a10-4a43-8bac-d745892049dd/kube-rbac-proxy/0.log"
Jul 12 19:15:20.825349 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.825335    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7wtg8_f7e4dbf6-2a10-4a43-8bac-d745892049dd/init-textfile/0.log"
Jul 12 19:15:20.937222 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.937198    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-xc6lx_f1b5a7de-cfef-4341-b852-add2f0d8ae2e/kube-rbac-proxy-main/0.log"
Jul 12 19:15:20.959226 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.959209    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-xc6lx_f1b5a7de-cfef-4341-b852-add2f0d8ae2e/kube-rbac-proxy-self/0.log"
Jul 12 19:15:20.985739 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:20.985721    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-xc6lx_f1b5a7de-cfef-4341-b852-add2f0d8ae2e/openshift-state-metrics/0.log"
Jul 12 19:15:21.021429 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.021409    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e6bc8957-a8df-47e6-8467-520bb945a6e2/prometheus/0.log"
Jul 12 19:15:21.041668 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.041649    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e6bc8957-a8df-47e6-8467-520bb945a6e2/config-reloader/0.log"
Jul 12 19:15:21.067790 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.067775    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e6bc8957-a8df-47e6-8467-520bb945a6e2/thanos-sidecar/0.log"
Jul 12 19:15:21.088014 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.087993    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e6bc8957-a8df-47e6-8467-520bb945a6e2/kube-rbac-proxy-web/0.log"
Jul 12 19:15:21.107471 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.107451    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e6bc8957-a8df-47e6-8467-520bb945a6e2/kube-rbac-proxy/0.log"
Jul 12 19:15:21.128089 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.128070    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e6bc8957-a8df-47e6-8467-520bb945a6e2/kube-rbac-proxy-thanos/0.log"
Jul 12 19:15:21.153492 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.153473    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e6bc8957-a8df-47e6-8467-520bb945a6e2/init-config-reloader/0.log"
Jul 12 19:15:21.190389 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.190340    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-jg6t7_3094dd13-57e9-445b-90e3-d22a097e4fec/prometheus-operator/0.log"
Jul 12 19:15:21.210479 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.210464    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-jg6t7_3094dd13-57e9-445b-90e3-d22a097e4fec/kube-rbac-proxy/0.log"
Jul 12 19:15:21.269059 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.269029    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-787fd96d66-mlrfx_609cdcb8-a619-420b-9ed8-38c2d34a9e99/telemeter-client/0.log"
Jul 12 19:15:21.295596 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.295581    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-787fd96d66-mlrfx_609cdcb8-a619-420b-9ed8-38c2d34a9e99/reload/0.log"
Jul 12 19:15:21.314610 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.314593    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-787fd96d66-mlrfx_609cdcb8-a619-420b-9ed8-38c2d34a9e99/kube-rbac-proxy/0.log"
Jul 12 19:15:21.356147 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.356130    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5cd8fc4c-6gtdm_8bb373d7-9362-4040-a204-6fdad64d4d8a/thanos-query/0.log"
Jul 12 19:15:21.374216 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.374195    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5cd8fc4c-6gtdm_8bb373d7-9362-4040-a204-6fdad64d4d8a/kube-rbac-proxy-web/0.log"
Jul 12 19:15:21.393543 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.393497    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5cd8fc4c-6gtdm_8bb373d7-9362-4040-a204-6fdad64d4d8a/kube-rbac-proxy/0.log"
Jul 12 19:15:21.411303 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.411286    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5cd8fc4c-6gtdm_8bb373d7-9362-4040-a204-6fdad64d4d8a/prom-label-proxy/0.log"
Jul 12 19:15:21.430000 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.429982    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5cd8fc4c-6gtdm_8bb373d7-9362-4040-a204-6fdad64d4d8a/kube-rbac-proxy-rules/0.log"
Jul 12 19:15:21.448239 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:21.448190    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5cd8fc4c-6gtdm_8bb373d7-9362-4040-a204-6fdad64d4d8a/kube-rbac-proxy-metrics/0.log"
Jul 12 19:15:22.897507 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.897475    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"]
Jul 12 19:15:22.897902 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.897876    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e49858-7bf1-42a9-a141-35c41ea6f3c8" containerName="main"
Jul 12 19:15:22.897902 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.897888    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e49858-7bf1-42a9-a141-35c41ea6f3c8" containerName="main"
Jul 12 19:15:22.897983 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.897978    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e49858-7bf1-42a9-a141-35c41ea6f3c8" containerName="main"
Jul 12 19:15:22.901754 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.901733    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:22.903931 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.903916    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h56g7\"/\"kube-root-ca.crt\""
Jul 12 19:15:22.904966 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.904943    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h56g7\"/\"openshift-service-ca.crt\""
Jul 12 19:15:22.904966 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.904962    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-h56g7\"/\"default-dockercfg-p4lkn\""
Jul 12 19:15:22.911018 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.910987    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"]
Jul 12 19:15:22.930817 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.930794    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tgr2\" (UniqueName: \"kubernetes.io/projected/8b4374e1-004d-4cc1-a881-5be8b5449a16-kube-api-access-4tgr2\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:22.930923 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.930839    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8b4374e1-004d-4cc1-a881-5be8b5449a16-proc\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:22.930923 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.930890    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b4374e1-004d-4cc1-a881-5be8b5449a16-sys\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:22.931002 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.930927    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b4374e1-004d-4cc1-a881-5be8b5449a16-lib-modules\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:22.931002 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:22.930967    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8b4374e1-004d-4cc1-a881-5be8b5449a16-podres\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:23.031306 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:23.031281    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8b4374e1-004d-4cc1-a881-5be8b5449a16-podres\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:23.031418 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:23.031322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tgr2\" (UniqueName: \"kubernetes.io/projected/8b4374e1-004d-4cc1-a881-5be8b5449a16-kube-api-access-4tgr2\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:23.031418 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:23.031357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8b4374e1-004d-4cc1-a881-5be8b5449a16-proc\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:23.031418 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:23.031383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b4374e1-004d-4cc1-a881-5be8b5449a16-sys\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:23.031418 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:23.031415    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b4374e1-004d-4cc1-a881-5be8b5449a16-lib-modules\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:23.031661 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:23.031444    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8b4374e1-004d-4cc1-a881-5be8b5449a16-podres\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:23.031661 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:23.031461    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b4374e1-004d-4cc1-a881-5be8b5449a16-sys\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:23.031661 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:23.031451    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8b4374e1-004d-4cc1-a881-5be8b5449a16-proc\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:23.031661 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:23.031542    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b4374e1-004d-4cc1-a881-5be8b5449a16-lib-modules\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:23.039346 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:23.039327    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tgr2\" (UniqueName: \"kubernetes.io/projected/8b4374e1-004d-4cc1-a881-5be8b5449a16-kube-api-access-4tgr2\") pod \"perf-node-gather-daemonset-6flpq\" (UID: \"8b4374e1-004d-4cc1-a881-5be8b5449a16\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:23.213468 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:23.213437    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:23.345905 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:23.345884    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"]
Jul 12 19:15:23.348151 ip-10-0-131-122 kubenswrapper[2578]: W0712 19:15:23.348124    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8b4374e1_004d_4cc1_a881_5be8b5449a16.slice/crio-b4ac2c67ecc423192bb11b8691daad13e550ead76a97fdfe5ebf46cd76393787 WatchSource:0}: Error finding container b4ac2c67ecc423192bb11b8691daad13e550ead76a97fdfe5ebf46cd76393787: Status 404 returned error can't find the container with id b4ac2c67ecc423192bb11b8691daad13e550ead76a97fdfe5ebf46cd76393787
Jul 12 19:15:23.811773 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:23.811745    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-656f5c87fb-hptk4_4917a3e2-1aaf-4e6f-bbc0-17f1dc8f72dc/console/0.log"
Jul 12 19:15:24.272751 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:24.272722    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq" event={"ID":"8b4374e1-004d-4cc1-a881-5be8b5449a16","Type":"ContainerStarted","Data":"fec9b2d95e4e7debcbe08ab5686daa95a831ef021f24a5321e579348df8305fc"}
Jul 12 19:15:24.273165 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:24.272772    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq" event={"ID":"8b4374e1-004d-4cc1-a881-5be8b5449a16","Type":"ContainerStarted","Data":"b4ac2c67ecc423192bb11b8691daad13e550ead76a97fdfe5ebf46cd76393787"}
Jul 12 19:15:24.273165 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:24.272890    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:24.287910 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:24.287865    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq" podStartSLOduration=2.287852547 podStartE2EDuration="2.287852547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 19:15:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:15:24.286796688 +0000 UTC m=+4899.486916363" watchObservedRunningTime="2026-07-12 19:15:24.287852547 +0000 UTC m=+4899.487972201"
Jul 12 19:15:25.497359 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:25.497327    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pmx65_19eadb03-b4c4-4f60-a60e-db68e797d2ad/dns/0.log"
Jul 12 19:15:25.518217 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:25.518194    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pmx65_19eadb03-b4c4-4f60-a60e-db68e797d2ad/kube-rbac-proxy/0.log"
Jul 12 19:15:25.589743 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:25.589721    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-89mdh_734454b5-e121-4543-ab79-ecfe0d985f96/dns-node-resolver/0.log"
Jul 12 19:15:26.220870 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:26.220845    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b2njf_b092918b-d322-40b1-b850-33b856bb1ba4/node-ca/0.log"
Jul 12 19:15:27.451644 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:27.451617    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-795d67df6d-rtl29_50471198-3766-43f5-acf6-f7f859a103fd/kube-auth-proxy/0.log"
Jul 12 19:15:27.592060 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:27.592035    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-tprz5_f7731e1d-da1f-4341-b6ea-436df7790893/payload-processing/0.log"
Jul 12 19:15:28.293768 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:28.293739    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hzhk4_eea7d684-a24b-4464-9bd6-6bca401e2791/serve-healthcheck-canary/0.log"
Jul 12 19:15:28.816643 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:28.816609    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8xmkt_3f14ee62-aa17-47d0-8e04-eb49058e0e91/kube-rbac-proxy/0.log"
Jul 12 19:15:28.835166 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:28.835143    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8xmkt_3f14ee62-aa17-47d0-8e04-eb49058e0e91/exporter/0.log"
Jul 12 19:15:28.853562 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:28.853541    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8xmkt_3f14ee62-aa17-47d0-8e04-eb49058e0e91/extractor/0.log"
Jul 12 19:15:30.287790 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:30.287763    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-6flpq"
Jul 12 19:15:31.180765 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:31.180740    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tpvlh_206a48fd-5615-432e-8d3c-f23279b775e9/manager/0.log"
Jul 12 19:15:31.284466 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:31.284438    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5dc7598fc6-txzwt_c8a6658f-4dfa-4c54-8f78-8155674e1e58/manager/0.log"
Jul 12 19:15:31.474135 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:31.474115    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv_2bd5a084-566d-436d-a7af-d65fd1c64ceb/manager/0.log"
Jul 12 19:15:31.494215 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:31.494197    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dbd4cc77c-fn8lv_2bd5a084-566d-436d-a7af-d65fd1c64ceb/copy-manifests/0.log"
Jul 12 19:15:33.018489 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:33.018465    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5484cf687f-5lnzc_c42b9eb8-29a5-4bcd-9c14-d4afb2da0538/manager/0.log"
Jul 12 19:15:33.067245 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:33.067220    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-8hn8g_672bf4e7-8189-4f59-b53d-7e6f792772e6/openshift-lws-operator/0.log"
Jul 12 19:15:33.471866 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:33.471835    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-795d67df6d-rtl29_50471198-3766-43f5-acf6-f7f859a103fd/kube-auth-proxy/0.log"
Jul 12 19:15:33.570866 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:33.570836    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-tprz5_f7731e1d-da1f-4341-b6ea-436df7790893/payload-processing/0.log"
Jul 12 19:15:35.019323 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:35.019295    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-795d67df6d-rtl29_50471198-3766-43f5-acf6-f7f859a103fd/kube-auth-proxy/0.log"
Jul 12 19:15:35.135417 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:35.135388    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-tprz5_f7731e1d-da1f-4341-b6ea-436df7790893/payload-processing/0.log"
Jul 12 19:15:39.350787 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:39.350758    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8wsqz_98dd2ed4-4bfb-4e7b-acae-e90695a81f22/kube-multus/0.log"
Jul 12 19:15:39.533539 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:39.533487    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fplg4_9ca436ac-91ae-4edb-975c-98ee1847a28a/kube-multus-additional-cni-plugins/0.log"
Jul 12 19:15:39.552795 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:39.552770    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fplg4_9ca436ac-91ae-4edb-975c-98ee1847a28a/egress-router-binary-copy/0.log"
Jul 12 19:15:39.571460 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:39.571437    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fplg4_9ca436ac-91ae-4edb-975c-98ee1847a28a/cni-plugins/0.log"
Jul 12 19:15:39.589114 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:39.589089    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fplg4_9ca436ac-91ae-4edb-975c-98ee1847a28a/bond-cni-plugin/0.log"
Jul 12 19:15:39.607657 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:39.607606    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fplg4_9ca436ac-91ae-4edb-975c-98ee1847a28a/routeoverride-cni/0.log"
Jul 12 19:15:39.627978 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:39.627959    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fplg4_9ca436ac-91ae-4edb-975c-98ee1847a28a/whereabouts-cni-bincopy/0.log"
Jul 12 19:15:39.645928 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:39.645911    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fplg4_9ca436ac-91ae-4edb-975c-98ee1847a28a/whereabouts-cni/0.log"
Jul 12 19:15:39.888907 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:39.888838    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-k4llc_4411fcad-827e-4fe9-9165-8eaba1e75226/network-metrics-daemon/0.log"
Jul 12 19:15:39.906874 ip-10-0-131-122 kubenswrapper[2578]: I0712 19:15:39.906851    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-k4llc_4411fcad-827e-4fe9-9165-8eaba1e75226/kube-rbac-proxy/0.log"