Jul 13 21:27:24.177227 ip-10-0-140-98 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 13 21:27:24.177239 ip-10-0-140-98 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 13 21:27:24.177247 ip-10-0-140-98 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 13 21:27:24.177523 ip-10-0-140-98 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 13 21:27:33.700993 ip-10-0-140-98 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 13 21:27:33.701009 ip-10-0-140-98 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 0445612896884cab974c61267c0eb2a9 --
Jul 13 21:29:16.679810 ip-10-0-140-98 systemd[1]: Starting Kubernetes Kubelet...
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.979312    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982686    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982698    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982702    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982706    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982709    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982712    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982715    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:29:18.484835 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982719    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:29:18.349226 ip-10-0-140-98 systemd[1]: Started Kubernetes Kubelet.
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982721    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982724    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982727    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982730    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982737    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982741    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982743    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982746    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982748    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982751    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982754    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982756    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982760    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982762    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982765    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982767    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982770    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982773    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982776    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:29:18.486205 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982780    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982783    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982787    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982790    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982792    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982795    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982798    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982801    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982806    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982811    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982814    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982816    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982819    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982822    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982825    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982828    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982831    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982834    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982836    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:29:18.487566 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982839    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982842    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982844    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982847    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982850    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982852    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982855    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982858    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982861    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982863    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982866    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982870    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982872    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982875    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982880    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982884    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982887    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982890    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982893    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982896    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:29:18.488557 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982898    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982901    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982904    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982907    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982910    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982913    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982915    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982918    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982921    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982923    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982926    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982929    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982932    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982934    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982937    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982942    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982945    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982948    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982951    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:29:18.489425 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982953    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982956    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982959    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982961    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982964    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982966    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982969    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982972    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982975    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982978    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982980    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982983    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982986    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.982988    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983712    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983721    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983724    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983727    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983730    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:29:18.490285 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983733    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983735    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983738    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983741    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983744    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983747    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983749    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983752    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983755    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983758    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983761    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983763    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983766    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983768    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983771    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983773    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983776    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983781    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983784    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:29:18.491365 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983787    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983790    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983794    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983799    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983802    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983805    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983807    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983810    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983813    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983816    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983818    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983821    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983823    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983826    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983829    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983831    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983834    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983836    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983839    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:29:18.492435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983841    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983845    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983848    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983850    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983854    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983857    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983860    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983862    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983865    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983867    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983870    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983875    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983878    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983881    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983884    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983886    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983890    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983893    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983896    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983898    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:29:18.493320 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983901    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983904    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983906    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983909    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983912    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983915    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983917    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983920    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983924    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983927    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983930    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983932    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983935    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983938    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983940    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983943    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983945    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983948    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983951    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983953    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:29:18.494394 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983956    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983959    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983961    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983965    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983967    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983971    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983974    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983977    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983980    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983982    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983985    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983988    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983991    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983994    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983996    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.983999    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984376    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984381    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984385    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984388    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:29:18.497339 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984391    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984393    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984396    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984400    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984404    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984407    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984410    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984413    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984416    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984419    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984421    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984424    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984427    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984432    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984434    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984438    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984441    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984443    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984446    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:29:18.498443 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984449    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984451    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984454    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984457    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984460    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984462    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984465    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984467    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984470    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984472    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984475    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984478    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984480    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984500    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984503    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984506    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984508    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984512    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984514    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984517    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:29:18.499520 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984519    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984522    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984524    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984527    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984529    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984532    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984536    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984539    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984545    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984549    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984552    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984555    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984558    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984561    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984564    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984567    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984570    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984573    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984576    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:29:18.501003 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984578    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984581    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984584    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984586    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984589    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984591    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984594    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984596    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984599    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984601    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984604    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984606    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984609    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984611    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984614    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984616    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984619    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984621    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984625    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984629    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:29:18.501873 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984631    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984635    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984638    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984640    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984643    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984647    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984650    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984653    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984656    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984658    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984664    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984667    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984669    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984672    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984675    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984677    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.984680    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984752    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984760    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984766    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984770    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 13 21:29:18.502776 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984774    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984777    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984781    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984786    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984789    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984792    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984795    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984798    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984801    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984804    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984809    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984811    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984816    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984819    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984824    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984827    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984830    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984833    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984837    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984841    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984844    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984847    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984851    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984854    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 13 21:29:18.503881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984857    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984860    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984863    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984866    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984870    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984873    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984877    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984880    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984883    2564 flags.go:64] FLAG: --enable-server="true"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984886    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984890    2564 flags.go:64] FLAG: --event-burst="100"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984894    2564 flags.go:64] FLAG: --event-qps="50"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984896    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984899    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984902    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984906    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984909    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984912    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984916    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984919    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984923    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984926    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984929    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984932    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984935    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 13 21:29:18.504993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984939    2564 flags.go:64] FLAG: --feature-gates=""
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984943    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984946    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984950    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984953    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984956    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984959    2564 flags.go:64] FLAG: --help="false"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984962    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984966    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984969    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984971    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984975    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984978    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984982    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984984    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984987    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984990    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984993    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984996    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.984999    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985002    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985005    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985008    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985011    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 13 21:29:18.506321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985014    2564 flags.go:64] FLAG: --lock-file=""
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985018    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985021    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985025    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985030    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985033    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985036    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985039    2564 flags.go:64] FLAG: --logging-format="text"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985042    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985046    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985049    2564 flags.go:64] FLAG: --manifest-url=""
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985051    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985055    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985058    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985062    2564 flags.go:64] FLAG: --max-pods="110"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985066    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985069    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985072    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985075    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985077    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985080    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985083    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985091    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985093    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985097    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 13 21:29:18.507388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985100    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985103    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985108    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985111    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985114    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985117    2564 flags.go:64] FLAG: --port="10250"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985120    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985123    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-090f3e5e121e83324"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985128    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985130    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985133    2564 flags.go:64] FLAG: --register-node="true"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985138    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985151    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985154    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985157    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985160    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985164    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985167    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985170    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985173    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985176    2564 flags.go:64] FLAG: --runonce="false"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985179    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985182    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985186    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985189    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 13 21:29:18.508885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985192    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985195    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985198    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985201    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985204    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985207    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985210    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985213    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985216    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985219    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985222    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985227    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985230    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985233    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985238    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985242    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985245    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985248    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985252    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985255    2564 flags.go:64] FLAG: --v="2"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985260    2564 flags.go:64] FLAG: --version="false"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985264    2564 flags.go:64] FLAG: --vmodule=""
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985268    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985271    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985385    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:29:18.509963 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985389    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985392    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985395    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985397    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985400    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985403    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985406    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985409    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985411    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985414    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985416    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985419    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985422    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985424    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985427    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985429    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985432    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985434    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985437    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985440    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:29:18.511464 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985442    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985445    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985449    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985452    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985455    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985459    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985461    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985464    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985467    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985469    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985472    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985474    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985477    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985479    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985496    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985499    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985502    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985505    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985507    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985510    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:29:18.512326 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985513    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985516    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985519    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985521    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985524    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985527    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985529    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985533    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985537    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985541    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985543    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985546    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985549    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985552    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985556    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985558    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985561    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985566    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985569    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:29:18.513391 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985571    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985574    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985576    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985580    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985584    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985587    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985590    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985592    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985595    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985598    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985600    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985603    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985605    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985609    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985612    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985615    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985619    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985623    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985625    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:29:18.514243 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985628    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985630    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985633    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985635    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985638    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985640    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985643    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985645    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985649    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985652    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985654    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985659    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985662    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985664    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985667    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985669    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985672    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985675    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985677    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:29:18.515312 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.985680    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.985685    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.993383    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.993401    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.993925    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.993934    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.993997    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994004    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994008    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994012    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994016    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994019    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994022    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994025    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:29:18.516183 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994029    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994032    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994036    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994039    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994042    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994045    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994047    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994052    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994055    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994059    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994063    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994069    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994073    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994076    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994080    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994084    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994087    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994089    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994093    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994097    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:29:18.517257 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994100    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994103    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994107    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994110    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994113    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994117    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994120    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994123    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994126    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994130    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994132    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994135    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994138    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994142    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994145    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994147    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994151    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994155    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994161    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994166    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:29:18.518343 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994169    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994172    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994175    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994180    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994182    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994186    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994189    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994191    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994194    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994197    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994200    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994204    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994209    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994213    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994215    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994219    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994222    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994225    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994228    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:29:18.520074 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994232    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994236    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994238    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994242    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994245    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994248    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994250    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994253    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994256    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994258    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994260    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994264    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994270    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994274    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994278    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994280    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994287    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994290    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994293    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994295    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:29:18.520947 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994299    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994302    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994304    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994307    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994311    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994314    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994317    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994322    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994325    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994328    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994332    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994336    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.994343    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994476    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994499    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:29:18.522234 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994503    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994509    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994514    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994518    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994522    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994525    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994529    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994532    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994534    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994541    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994545    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994548    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994551    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994554    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994559    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994562    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994565    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994568    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994570    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:29:18.523085 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994574    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994577    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994580    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994582    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994585    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994589    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994592    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994595    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994599    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994601    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994604    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994607    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994611    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994613    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994616    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994619    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994622    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994624    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994627    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994630    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:29:18.523960 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994633    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994636    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994640    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994645    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994649    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994653    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994656    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994661    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994663    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994666    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994668    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994671    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994675    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994679    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994681    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994684    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994688    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994690    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994693    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994695    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:29:18.524848 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994699    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994701    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994704    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994707    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994710    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994713    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994715    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994718    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994722    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994724    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994726    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994729    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994732    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994735    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994739    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994742    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994745    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994749    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994751    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994756    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:29:18.527165 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994759    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994762    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994765    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994768    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994772    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994776    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994780    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994783    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994786    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994789    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994793    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994796    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994798    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994802    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994805    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994808    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994810    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:29:18.528572 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:16.994814    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:29:18.529853 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.994819    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 21:29:18.529853 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.994941    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 13 21:29:18.529853 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.996988    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 13 21:29:18.529853 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.998036    2564 server.go:1019] "Starting client certificate rotation"
Jul 13 21:29:18.529853 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.998140    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 21:29:18.529853 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:16.998715    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 21:29:18.529853 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:17.000543    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 21:29:18.529853 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:17.001730    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 21:29:18.529853 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:17.011289    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 13 21:29:18.529853 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:17.013388    2564 log.go:25] "Validated CRI v1 image API"
Jul 13 21:29:18.529853 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:17.014221    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 13 21:29:18.529853 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:17.016306    2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d44b4605-8e46-4392-997a-a45cd0875fb5:/dev/nvme0n1p4 f04e1682-759f-49f5-b537-6c0d528db99e:/dev/nvme0n1p3]
Jul 13 21:29:18.530506 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:17.016325    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 13 21:29:18.762696 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:17.024825    2564 manager.go:216] Machine: {Timestamp:2026-07-13 21:29:17.023633145 +0000 UTC m=+0.250843681 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100045 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec263f82745038c342a74e676afa4e1c SystemUUID:ec263f82-7450-38c3-42a7-4e676afa4e1c BootID:04456128-9688-4cab-974c-61267c0eb2a9 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:91:d3:d1:3e:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:91:d3:d1:3e:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:b8:a8:3b:a5:34 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 13 21:29:18.762696 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.330284    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 13 21:29:18.762696 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.330433    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 13 21:29:18.762696 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.331043    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 21:29:18.762696 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.332977    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 13 21:29:18.762696 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.333010    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-98.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 13 21:29:18.762696 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.333687    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.333697    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.333705    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.334214    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.335012    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.338715    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.338833    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.338862    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.338873    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.338887    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.338895    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.338904    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.339346    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.339369    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.341110    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.342773    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.342803    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.343469    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.343487    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.343497    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.343503    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.343515    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.343522    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.343531    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.343536    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.343543    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.343550    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.343559    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.343566    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.344638    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.344662    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.348190    2564 server.go:1268] "Started kubelet"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.348260    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.348279    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.348368    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.349498    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.351321    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-98.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.354156    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.353903    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 13 21:29:18.763105 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.352504    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-98.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.350620    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.357207    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-98.ec2.internal.18c1f6aedf65efd2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-98.ec2.internal,UID:ip-10-0-140-98.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-98.ec2.internal,},FirstTimestamp:2026-07-13 21:29:18.34816917 +0000 UTC m=+0.270218077,LastTimestamp:2026-07-13 21:29:18.34816917 +0000 UTC m=+0.270218077,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-98.ec2.internal,}"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.358471    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.358478    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.358672    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-98.ec2.internal\" not found"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.359259    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.359261    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.359291    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.359362    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.359365    2564 factory.go:55] Registering systemd factory
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.359383    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.359374    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.359663    2564 factory.go:153] Registering CRI-O factory
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.359687    2564 factory.go:223] Registration of the crio container factory successfully
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.359732    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.359758    2564 factory.go:103] Registering Raw factory
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.359780    2564 manager.go:1196] Started watching for new ooms in manager
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.360262    2564 manager.go:349] Starting recovery of all containers
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.366844    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-98.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.367562    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.375402    2564 manager.go:354] Recovery completed
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.377805    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m2k9k"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.379507    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.381384    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.381411    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.381421    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.381952    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.381960    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.381962    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.381977    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.384079    2564 policy_none.go:49] "None policy: Start"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.384095    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 13 21:29:18.764252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.384107    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.385739    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.385741    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m2k9k"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.385761    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.385771    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.385844    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-98.ec2.internal.18c1f6aee160f2b2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-98.ec2.internal,UID:ip-10-0-140-98.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-98.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-98.ec2.internal,},FirstTimestamp:2026-07-13 21:29:18.381396658 +0000 UTC m=+0.303445568,LastTimestamp:2026-07-13 21:29:18.381396658 +0000 UTC m=+0.303445568,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-98.ec2.internal,}"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.386622    2564 policy_none.go:47] "Start"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.431322    2564 manager.go:337] "Starting Device Plugin manager"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.431581    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.431592    2564 server.go:87] "Starting device plugin registration server"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.431834    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.431846    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.431928    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.432001    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.432007    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.432743    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.432792    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-98.ec2.internal\" not found"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.465188    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.466456    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.466500    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.466530    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.466571    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.468749    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.532916    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.533895    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.533929    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.533943    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.533968    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.540147    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.540169    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-98.ec2.internal\": node \"ip-10-0-140-98.ec2.internal\" not found"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.565106    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-98.ec2.internal\" not found"
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.567137    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-98.ec2.internal"]
Jul 13 21:29:18.765190 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.567206    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.568714    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.568739    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.568752    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.571084    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.571189    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.571226    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.571834    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.571854    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.571881    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.571898    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.571902    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.571915    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.574106    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.574129    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.574851    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.574874    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.574884    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.594623    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-98.ec2.internal\" not found" node="ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.597711    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-98.ec2.internal\" not found" node="ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.660083    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be4b5ee886bda5efae9a7f3c00ec6ced-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal\" (UID: \"be4b5ee886bda5efae9a7f3c00ec6ced\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.660109    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/09e44c48acdcebe81c75239f1dc1647b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-98.ec2.internal\" (UID: \"09e44c48acdcebe81c75239f1dc1647b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.660125    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/be4b5ee886bda5efae9a7f3c00ec6ced-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal\" (UID: \"be4b5ee886bda5efae9a7f3c00ec6ced\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.666104    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-98.ec2.internal\" not found"
Jul 13 21:29:18.766118 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.760241    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/be4b5ee886bda5efae9a7f3c00ec6ced-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal\" (UID: \"be4b5ee886bda5efae9a7f3c00ec6ced\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.766825 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.760295    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be4b5ee886bda5efae9a7f3c00ec6ced-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal\" (UID: \"be4b5ee886bda5efae9a7f3c00ec6ced\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.766825 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.760312    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/09e44c48acdcebe81c75239f1dc1647b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-98.ec2.internal\" (UID: \"09e44c48acdcebe81c75239f1dc1647b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.766825 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.760338    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/09e44c48acdcebe81c75239f1dc1647b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-98.ec2.internal\" (UID: \"09e44c48acdcebe81c75239f1dc1647b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.766825 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.760264    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/be4b5ee886bda5efae9a7f3c00ec6ced-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal\" (UID: \"be4b5ee886bda5efae9a7f3c00ec6ced\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.766825 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.760392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be4b5ee886bda5efae9a7f3c00ec6ced-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal\" (UID: \"be4b5ee886bda5efae9a7f3c00ec6ced\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.766825 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.766340    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-98.ec2.internal\" not found"
Jul 13 21:29:18.960225 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.867070    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-98.ec2.internal\" not found"
Jul 13 21:29:18.960225 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.897234    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:18.960225 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:18.900390    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:19.154237 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:18.967760    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-98.ec2.internal\" not found"
Jul 13 21:29:19.154237 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:19.068332    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-98.ec2.internal\" not found"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:19.168798    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-98.ec2.internal\" not found"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.226563    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:19.269532    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-98.ec2.internal\" not found"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.303677    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.303828    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.303828    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.344075    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.361895    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.361932    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.375110    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.375328    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.375880    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-98.ec2.internal"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.387068    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.390233    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 21:24:18 +0000 UTC" deadline="2028-02-09 13:39:39.672999215 +0000 UTC"
Jul 13 21:29:19.393883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.390255    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13816h10m20.282746584s"
Jul 13 21:29:19.562980 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.409365    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s7p8n"
Jul 13 21:29:19.562980 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.419479    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s7p8n"
Jul 13 21:29:19.635092 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:19.635051    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e44c48acdcebe81c75239f1dc1647b.slice/crio-8427b9def3bb05aad02d54364b1345b62e4caa573c13b74546b3c341ef0f320c WatchSource:0}: Error finding container 8427b9def3bb05aad02d54364b1345b62e4caa573c13b74546b3c341ef0f320c: Status 404 returned error can't find the container with id 8427b9def3bb05aad02d54364b1345b62e4caa573c13b74546b3c341ef0f320c
Jul 13 21:29:19.636098 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.636075    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:29:19.639556 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:19.639530    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe4b5ee886bda5efae9a7f3c00ec6ced.slice/crio-499661dce2d0814f67b2b0dec9541b1d56b3b3236f349be4f17e72c38ca3ad85 WatchSource:0}: Error finding container 499661dce2d0814f67b2b0dec9541b1d56b3b3236f349be4f17e72c38ca3ad85: Status 404 returned error can't find the container with id 499661dce2d0814f67b2b0dec9541b1d56b3b3236f349be4f17e72c38ca3ad85
Jul 13 21:29:19.640328 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:19.640315    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:29:20.339320 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.339289    2564 apiserver.go:52] "Watching apiserver"
Jul 13 21:29:20.345359 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.345331    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 13 21:29:20.345751 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.345718    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-pt6c7","openshift-multus/multus-nhhkd","openshift-network-operator/iptables-alerter-kt42z","openshift-ovn-kubernetes/ovnkube-node-xv5wb","kube-system/global-pull-secret-syncer-nkmbp","kube-system/kube-apiserver-proxy-ip-10-0-140-98.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw","openshift-image-registry/node-ca-4v4mk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal","openshift-multus/multus-additional-cni-plugins-qlllp","openshift-multus/network-metrics-daemon-fjzfw","openshift-network-diagnostics/network-check-target-td5gj","kube-system/konnectivity-agent-rrzxh","openshift-cluster-node-tuning-operator/tuned-9x42s"]
Jul 13 21:29:20.348369 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.348348    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.350532 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.350506    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 13 21:29:20.350664 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.350598    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-smdsh\""
Jul 13 21:29:20.350938 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.350761    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 13 21:29:20.350938 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.350789    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 13 21:29:20.350938 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.350937    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rrzxh"
Jul 13 21:29:20.352921 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.352904    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 13 21:29:20.353029 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.353009    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 13 21:29:20.353029 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.353023    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-llx9z\""
Jul 13 21:29:20.353211 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.353143    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kt42z"
Jul 13 21:29:20.355049 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.355023    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:29:20.355497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.355319    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 13 21:29:20.355497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.355428    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5gb6l\""
Jul 13 21:29:20.355497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.355485    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 13 21:29:20.357283 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.355789    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.358279 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.358043    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p2qx7\""
Jul 13 21:29:20.358279 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.358081    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 13 21:29:20.358279 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.358089    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:20.358279 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:20.358159    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:20.358279 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.358220    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 13 21:29:20.358642 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.358618    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 13 21:29:20.358734 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.358720    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 13 21:29:20.358800 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.358623    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 13 21:29:20.358800 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.358792    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 13 21:29:20.360362 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.360328    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.362379 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.362229    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5bhx2\""
Jul 13 21:29:20.362379 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.362232    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:29:20.362379 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.362271    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 13 21:29:20.362635 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.362621    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4v4mk"
Jul 13 21:29:20.364568 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.364400    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 13 21:29:20.364714 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.364694    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 13 21:29:20.364803 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.364722    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nn45c\""
Jul 13 21:29:20.364867 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.364855    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 13 21:29:20.365046 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.365026    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pt6c7"
Jul 13 21:29:20.366989 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.366971    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 13 21:29:20.366989 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.366985    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 13 21:29:20.367134 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.367031    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sjfbl\""
Jul 13 21:29:20.367462 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.367445    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.369331 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.369312    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 13 21:29:20.369428 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.369314    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bc9kb\""
Jul 13 21:29:20.369428 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.369393    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 13 21:29:20.369532 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.369521    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 13 21:29:20.369800 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.369785    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 13 21:29:20.370081 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.370064    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.371885 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.371868    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 13 21:29:20.371978 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.371868    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8pp7f\""
Jul 13 21:29:20.371978 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.371933    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 13 21:29:20.372699 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.372682    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:20.372787 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:20.372756    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:20.374970 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.374954    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:20.375051 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:20.375011    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:20.376873 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.376842    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-kubelet\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.376969 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.376885    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-systemd-units\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.376969 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.376905    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-slash\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.376969 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.376920    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-var-lib-openvswitch\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.376969 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.376951    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-log-socket\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.377172 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.376996    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-cni-netd\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.377172 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377023    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/abaedb55-25f3-46db-a793-cc0a4f71ddd3-konnectivity-ca\") pod \"konnectivity-agent-rrzxh\" (UID: \"abaedb55-25f3-46db-a793-cc0a4f71ddd3\") " pod="kube-system/konnectivity-agent-rrzxh"
Jul 13 21:29:20.377172 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377050    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-run-openvswitch\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.377172 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377071    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/15165918-15d0-4f74-b300-931b0c8d67f5-ovnkube-config\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.377172 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377101    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj5pd\" (UniqueName: \"kubernetes.io/projected/15165918-15d0-4f74-b300-931b0c8d67f5-kube-api-access-gj5pd\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.377172 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377128    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:20.377172 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377155    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-kubernetes\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.377497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377181    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-sysctl-d\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.377497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377246    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-etc-selinux\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.377497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377271    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-run-ovn\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.377497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377294    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/15165918-15d0-4f74-b300-931b0c8d67f5-env-overrides\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.377497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377318    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/20215437-0510-495e-8a20-3a272973cdfd-etc-tuned\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.377497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377344    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.377497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377384    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-socket-dir\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.377497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377406    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-sys-fs\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.377497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377427    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-run-ovn-kubernetes\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.377497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377446    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/15165918-15d0-4f74-b300-931b0c8d67f5-ovn-node-metrics-cert\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.377497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377479    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd9888e0-94ba-4da2-b458-8bad7d8a136b-iptables-alerter-script\") pod \"iptables-alerter-kt42z\" (UID: \"fd9888e0-94ba-4da2-b458-8bad7d8a136b\") " pod="openshift-network-operator/iptables-alerter-kt42z"
Jul 13 21:29:20.377497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377493    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd9888e0-94ba-4da2-b458-8bad7d8a136b-host-slash\") pod \"iptables-alerter-kt42z\" (UID: \"fd9888e0-94ba-4da2-b458-8bad7d8a136b\") " pod="openshift-network-operator/iptables-alerter-kt42z"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377531    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-run-netns\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377564    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-run-systemd\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377587    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-etc-openvswitch\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377610    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-cni-bin\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377700    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-sysconfig\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377738    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377765    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-modprobe-d\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377779    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-sysctl-conf\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377792    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-systemd\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377807    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-run\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377829    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-sys\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377849    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-lib-modules\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377875    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20215437-0510-495e-8a20-3a272973cdfd-tmp\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.378058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377895    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-device-dir\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.378590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377909    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/abaedb55-25f3-46db-a793-cc0a4f71ddd3-agent-certs\") pod \"konnectivity-agent-rrzxh\" (UID: \"abaedb55-25f3-46db-a793-cc0a4f71ddd3\") " pod="kube-system/konnectivity-agent-rrzxh"
Jul 13 21:29:20.378590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377924    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg2lk\" (UniqueName: \"kubernetes.io/projected/fd9888e0-94ba-4da2-b458-8bad7d8a136b-kube-api-access-pg2lk\") pod \"iptables-alerter-kt42z\" (UID: \"fd9888e0-94ba-4da2-b458-8bad7d8a136b\") " pod="openshift-network-operator/iptables-alerter-kt42z"
Jul 13 21:29:20.378590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377943    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-node-log\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.378590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.377989    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0cb23a38-df98-4abc-9ca3-43fe622a34c7-kubelet-config\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:20.378590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.378006    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-var-lib-kubelet\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.378590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.378030    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-registration-dir\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.378590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.378051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvsrt\" (UniqueName: \"kubernetes.io/projected/7af3cbf7-78c5-428d-a699-f0341cbd04d6-kube-api-access-qvsrt\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.378590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.378067    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/15165918-15d0-4f74-b300-931b0c8d67f5-ovnkube-script-lib\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.378590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.378081    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0cb23a38-df98-4abc-9ca3-43fe622a34c7-dbus\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:20.378590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.378120    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-host\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.378590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.378156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q7dz\" (UniqueName: \"kubernetes.io/projected/20215437-0510-495e-8a20-3a272973cdfd-kube-api-access-4q7dz\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.422323 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.421950    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:24:19 +0000 UTC" deadline="2028-03-28 03:13:56.109932478 +0000 UTC"
Jul 13 21:29:20.422323 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.422322    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14957h44m35.687618287s"
Jul 13 21:29:20.460317 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.460288    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 13 21:29:20.471778 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.471725    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal" event={"ID":"be4b5ee886bda5efae9a7f3c00ec6ced","Type":"ContainerStarted","Data":"499661dce2d0814f67b2b0dec9541b1d56b3b3236f349be4f17e72c38ca3ad85"}
Jul 13 21:29:20.472939 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.472904    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-98.ec2.internal" event={"ID":"09e44c48acdcebe81c75239f1dc1647b","Type":"ContainerStarted","Data":"8427b9def3bb05aad02d54364b1345b62e4caa573c13b74546b3c341ef0f320c"}
Jul 13 21:29:20.479283 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479254    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd9888e0-94ba-4da2-b458-8bad7d8a136b-iptables-alerter-script\") pod \"iptables-alerter-kt42z\" (UID: \"fd9888e0-94ba-4da2-b458-8bad7d8a136b\") " pod="openshift-network-operator/iptables-alerter-kt42z"
Jul 13 21:29:20.479399 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479295    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd9888e0-94ba-4da2-b458-8bad7d8a136b-host-slash\") pod \"iptables-alerter-kt42z\" (UID: \"fd9888e0-94ba-4da2-b458-8bad7d8a136b\") " pod="openshift-network-operator/iptables-alerter-kt42z"
Jul 13 21:29:20.479399 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479321    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-run-netns\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.479399 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479346    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-sysctl-conf\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.479399 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479369    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-run\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.479399 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479377    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd9888e0-94ba-4da2-b458-8bad7d8a136b-host-slash\") pod \"iptables-alerter-kt42z\" (UID: \"fd9888e0-94ba-4da2-b458-8bad7d8a136b\") " pod="openshift-network-operator/iptables-alerter-kt42z"
Jul 13 21:29:20.479399 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479391    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-sys\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.479681 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479433    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-867pr\" (UniqueName: \"kubernetes.io/projected/c8726c19-73c9-4166-97a2-88f09f2bb28c-kube-api-access-867pr\") pod \"node-ca-4v4mk\" (UID: \"c8726c19-73c9-4166-97a2-88f09f2bb28c\") " pod="openshift-image-registry/node-ca-4v4mk"
Jul 13 21:29:20.479681 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479460    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-run\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.479681 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479467    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-modprobe-d\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.479681 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479468    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-run-netns\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.479681 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479530    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-sys\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.479681 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20215437-0510-495e-8a20-3a272973cdfd-tmp\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.479681 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479672    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e70ee258-310d-4f06-9908-afc7a27684da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.479983 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-sysctl-conf\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.479983 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479702    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwnj4\" (UniqueName: \"kubernetes.io/projected/53433330-3378-4829-be11-5a219e621ef4-kube-api-access-cwnj4\") pod \"node-resolver-pt6c7\" (UID: \"53433330-3378-4829-be11-5a219e621ef4\") " pod="openshift-dns/node-resolver-pt6c7"
Jul 13 21:29:20.479983 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479729    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-os-release\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.479983 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479747    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-modprobe-d\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.479983 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479752    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-var-lib-kubelet\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.479983 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479802    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-run-multus-certs\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.479983 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/abaedb55-25f3-46db-a793-cc0a4f71ddd3-agent-certs\") pod \"konnectivity-agent-rrzxh\" (UID: \"abaedb55-25f3-46db-a793-cc0a4f71ddd3\") " pod="kube-system/konnectivity-agent-rrzxh"
Jul 13 21:29:20.479983 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479866    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-node-log\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.479983 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479893    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0cb23a38-df98-4abc-9ca3-43fe622a34c7-kubelet-config\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:20.479983 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479969    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd9888e0-94ba-4da2-b458-8bad7d8a136b-iptables-alerter-script\") pod \"iptables-alerter-kt42z\" (UID: \"fd9888e0-94ba-4da2-b458-8bad7d8a136b\") " pod="openshift-network-operator/iptables-alerter-kt42z"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e70ee258-310d-4f06-9908-afc7a27684da-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480017    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5a578cc-d952-452d-bc46-fa37ed934581-cni-binary-copy\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.479987    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480052    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-node-log\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480095    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-hostroot\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480122    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-multus-conf-dir\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480156    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0cb23a38-df98-4abc-9ca3-43fe622a34c7-kubelet-config\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480214    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srpvm\" (UniqueName: \"kubernetes.io/projected/f5a578cc-d952-452d-bc46-fa37ed934581-kube-api-access-srpvm\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvsrt\" (UniqueName: \"kubernetes.io/projected/7af3cbf7-78c5-428d-a699-f0341cbd04d6-kube-api-access-qvsrt\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/15165918-15d0-4f74-b300-931b0c8d67f5-ovnkube-script-lib\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480301    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0cb23a38-df98-4abc-9ca3-43fe622a34c7-dbus\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480324    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q7dz\" (UniqueName: \"kubernetes.io/projected/20215437-0510-495e-8a20-3a272973cdfd-kube-api-access-4q7dz\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480347    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53433330-3378-4829-be11-5a219e621ef4-hosts-file\") pod \"node-resolver-pt6c7\" (UID: \"53433330-3378-4829-be11-5a219e621ef4\") " pod="openshift-dns/node-resolver-pt6c7"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480374    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-var-lib-cni-bin\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.480438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480397    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-slash\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480418    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-cni-netd\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480487    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0cb23a38-df98-4abc-9ca3-43fe622a34c7-dbus\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480497    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-slash\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480541    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-cni-netd\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480560    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c8726c19-73c9-4166-97a2-88f09f2bb28c-host\") pod \"node-ca-4v4mk\" (UID: \"c8726c19-73c9-4166-97a2-88f09f2bb28c\") " pod="openshift-image-registry/node-ca-4v4mk"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480599    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e70ee258-310d-4f06-9908-afc7a27684da-os-release\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/15165918-15d0-4f74-b300-931b0c8d67f5-ovnkube-config\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480671    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj5pd\" (UniqueName: \"kubernetes.io/projected/15165918-15d0-4f74-b300-931b0c8d67f5-kube-api-access-gj5pd\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480696    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-sysctl-d\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480721    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e70ee258-310d-4f06-9908-afc7a27684da-system-cni-dir\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480767    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-multus-socket-dir-parent\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480794    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-etc-selinux\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480818    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-etc-kubernetes\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.481151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480845    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480868    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-socket-dir\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480909    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e70ee258-310d-4f06-9908-afc7a27684da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480915    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-sysctl-d\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480927    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e70ee258-310d-4f06-9908-afc7a27684da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480944    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-run-systemd\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480964    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-etc-openvswitch\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480973    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480986    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-cni-bin\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.480989    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/15165918-15d0-4f74-b300-931b0c8d67f5-ovnkube-script-lib\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481000    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-etc-selinux\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481013    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-sysconfig\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481046    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-cni-bin\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481052    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-sysconfig\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.481865 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481089    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-systemd\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481073    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-socket-dir\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481090    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-etc-openvswitch\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481121    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-run-systemd\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481122    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-lib-modules\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m7fh\" (UniqueName: \"kubernetes.io/projected/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-kube-api-access-8m7fh\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481179    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c8726c19-73c9-4166-97a2-88f09f2bb28c-serviceca\") pod \"node-ca-4v4mk\" (UID: \"c8726c19-73c9-4166-97a2-88f09f2bb28c\") " pod="openshift-image-registry/node-ca-4v4mk"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481126    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-systemd\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481204    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481225    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-lib-modules\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481228    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-system-cni-dir\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481253    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-device-dir\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481293    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg2lk\" (UniqueName: \"kubernetes.io/projected/fd9888e0-94ba-4da2-b458-8bad7d8a136b-kube-api-access-pg2lk\") pod \"iptables-alerter-kt42z\" (UID: \"fd9888e0-94ba-4da2-b458-8bad7d8a136b\") " pod="openshift-network-operator/iptables-alerter-kt42z"
Jul 13 21:29:20.482513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481317    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-var-lib-kubelet\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481343    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53433330-3378-4829-be11-5a219e621ef4-tmp-dir\") pod \"node-resolver-pt6c7\" (UID: \"53433330-3378-4829-be11-5a219e621ef4\") " pod="openshift-dns/node-resolver-pt6c7"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481355    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-device-dir\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481366    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-multus-cni-dir\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481379    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-var-lib-kubelet\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481390    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-var-lib-cni-multus\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481416    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f5a578cc-d952-452d-bc46-fa37ed934581-multus-daemon-config\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481441    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-registration-dir\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481465    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-host\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481474    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-registration-dir\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481487    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e70ee258-310d-4f06-9908-afc7a27684da-cnibin\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481505    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-host\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p6xg\" (UniqueName: \"kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg\") pod \"network-check-target-td5gj\" (UID: \"b25ee3f1-7546-49f6-bb7c-6f97b33574f8\") " pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481539    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-kubelet\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.483116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481562    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-systemd-units\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481584    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-var-lib-openvswitch\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481596    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-kubelet\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481607    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-log-socket\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481629    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-cnibin\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481644    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-systemd-units\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481670    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-var-lib-openvswitch\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481678    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/abaedb55-25f3-46db-a793-cc0a4f71ddd3-konnectivity-ca\") pod \"konnectivity-agent-rrzxh\" (UID: \"abaedb55-25f3-46db-a793-cc0a4f71ddd3\") " pod="kube-system/konnectivity-agent-rrzxh"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481683    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/15165918-15d0-4f74-b300-931b0c8d67f5-ovnkube-config\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481703    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-run-openvswitch\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481698    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-log-socket\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481732    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481751    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-run-openvswitch\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481757    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-kubernetes\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.483590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481782    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481806    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-run-k8s-cni-cncf-io\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-run-ovn\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:20.481833    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481844    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20215437-0510-495e-8a20-3a272973cdfd-etc-kubernetes\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481853    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/15165918-15d0-4f74-b300-931b0c8d67f5-env-overrides\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481875    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/20215437-0510-495e-8a20-3a272973cdfd-etc-tuned\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481886    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-run-ovn\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:20.481902    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret podName:0cb23a38-df98-4abc-9ca3-43fe622a34c7 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:20.981882779 +0000 UTC m=+2.903931695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret") pod "global-pull-secret-syncer-nkmbp" (UID: "0cb23a38-df98-4abc-9ca3-43fe622a34c7") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481921    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-sys-fs\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481956    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-run-ovn-kubernetes\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481969    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7af3cbf7-78c5-428d-a699-f0341cbd04d6-sys-fs\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.481981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/15165918-15d0-4f74-b300-931b0c8d67f5-ovn-node-metrics-cert\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.482007    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6m8r\" (UniqueName: \"kubernetes.io/projected/e70ee258-310d-4f06-9908-afc7a27684da-kube-api-access-p6m8r\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.484159 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.482009    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15165918-15d0-4f74-b300-931b0c8d67f5-host-run-ovn-kubernetes\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.484734 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.482032    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-run-netns\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.484734 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.482240    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/15165918-15d0-4f74-b300-931b0c8d67f5-env-overrides\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.484734 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.482735    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/abaedb55-25f3-46db-a793-cc0a4f71ddd3-konnectivity-ca\") pod \"konnectivity-agent-rrzxh\" (UID: \"abaedb55-25f3-46db-a793-cc0a4f71ddd3\") " pod="kube-system/konnectivity-agent-rrzxh"
Jul 13 21:29:20.484734 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.484012    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20215437-0510-495e-8a20-3a272973cdfd-tmp\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.484734 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.484239    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/abaedb55-25f3-46db-a793-cc0a4f71ddd3-agent-certs\") pod \"konnectivity-agent-rrzxh\" (UID: \"abaedb55-25f3-46db-a793-cc0a4f71ddd3\") " pod="kube-system/konnectivity-agent-rrzxh"
Jul 13 21:29:20.484734 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.484309    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/20215437-0510-495e-8a20-3a272973cdfd-etc-tuned\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.484734 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.484475    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/15165918-15d0-4f74-b300-931b0c8d67f5-ovn-node-metrics-cert\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.487953 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.487887    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q7dz\" (UniqueName: \"kubernetes.io/projected/20215437-0510-495e-8a20-3a272973cdfd-kube-api-access-4q7dz\") pod \"tuned-9x42s\" (UID: \"20215437-0510-495e-8a20-3a272973cdfd\") " pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.488922 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.488894    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg2lk\" (UniqueName: \"kubernetes.io/projected/fd9888e0-94ba-4da2-b458-8bad7d8a136b-kube-api-access-pg2lk\") pod \"iptables-alerter-kt42z\" (UID: \"fd9888e0-94ba-4da2-b458-8bad7d8a136b\") " pod="openshift-network-operator/iptables-alerter-kt42z"
Jul 13 21:29:20.489305 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.489282    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj5pd\" (UniqueName: \"kubernetes.io/projected/15165918-15d0-4f74-b300-931b0c8d67f5-kube-api-access-gj5pd\") pod \"ovnkube-node-xv5wb\" (UID: \"15165918-15d0-4f74-b300-931b0c8d67f5\") " pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.489382 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.489316    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvsrt\" (UniqueName: \"kubernetes.io/projected/7af3cbf7-78c5-428d-a699-f0341cbd04d6-kube-api-access-qvsrt\") pod \"aws-ebs-csi-driver-node-glccw\" (UID: \"7af3cbf7-78c5-428d-a699-f0341cbd04d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.582431 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582387    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-multus-conf-dir\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.582431 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582436    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srpvm\" (UniqueName: \"kubernetes.io/projected/f5a578cc-d952-452d-bc46-fa37ed934581-kube-api-access-srpvm\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582465    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53433330-3378-4829-be11-5a219e621ef4-hosts-file\") pod \"node-resolver-pt6c7\" (UID: \"53433330-3378-4829-be11-5a219e621ef4\") " pod="openshift-dns/node-resolver-pt6c7"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582490    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-var-lib-cni-bin\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582487    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-multus-conf-dir\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582516    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c8726c19-73c9-4166-97a2-88f09f2bb28c-host\") pod \"node-ca-4v4mk\" (UID: \"c8726c19-73c9-4166-97a2-88f09f2bb28c\") " pod="openshift-image-registry/node-ca-4v4mk"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e70ee258-310d-4f06-9908-afc7a27684da-os-release\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582560    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c8726c19-73c9-4166-97a2-88f09f2bb28c-host\") pod \"node-ca-4v4mk\" (UID: \"c8726c19-73c9-4166-97a2-88f09f2bb28c\") " pod="openshift-image-registry/node-ca-4v4mk"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582573    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-var-lib-cni-bin\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582566    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e70ee258-310d-4f06-9908-afc7a27684da-system-cni-dir\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582603    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e70ee258-310d-4f06-9908-afc7a27684da-system-cni-dir\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582634    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-multus-socket-dir-parent\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53433330-3378-4829-be11-5a219e621ef4-hosts-file\") pod \"node-resolver-pt6c7\" (UID: \"53433330-3378-4829-be11-5a219e621ef4\") " pod="openshift-dns/node-resolver-pt6c7"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582671    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-etc-kubernetes\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582636    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e70ee258-310d-4f06-9908-afc7a27684da-os-release\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582694    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e70ee258-310d-4f06-9908-afc7a27684da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.583273 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582716    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-multus-socket-dir-parent\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582724    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e70ee258-310d-4f06-9908-afc7a27684da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582732    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-etc-kubernetes\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582754    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m7fh\" (UniqueName: \"kubernetes.io/projected/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-kube-api-access-8m7fh\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582778    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c8726c19-73c9-4166-97a2-88f09f2bb28c-serviceca\") pod \"node-ca-4v4mk\" (UID: \"c8726c19-73c9-4166-97a2-88f09f2bb28c\") " pod="openshift-image-registry/node-ca-4v4mk"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582802    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-system-cni-dir\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53433330-3378-4829-be11-5a219e621ef4-tmp-dir\") pod \"node-resolver-pt6c7\" (UID: \"53433330-3378-4829-be11-5a219e621ef4\") " pod="openshift-dns/node-resolver-pt6c7"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582849    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-multus-cni-dir\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582853    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e70ee258-310d-4f06-9908-afc7a27684da-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582874    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-var-lib-cni-multus\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582893    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-system-cni-dir\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582912    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-var-lib-cni-multus\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582912    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f5a578cc-d952-452d-bc46-fa37ed934581-multus-daemon-config\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582943    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e70ee258-310d-4f06-9908-afc7a27684da-cnibin\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.582978    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p6xg\" (UniqueName: \"kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg\") pod \"network-check-target-td5gj\" (UID: \"b25ee3f1-7546-49f6-bb7c-6f97b33574f8\") " pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:20.584096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583006    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-cnibin\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583045    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-multus-cni-dir\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583064    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-cnibin\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583096    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e70ee258-310d-4f06-9908-afc7a27684da-cnibin\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583147    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583170    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-run-k8s-cni-cncf-io\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583201    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6m8r\" (UniqueName: \"kubernetes.io/projected/e70ee258-310d-4f06-9908-afc7a27684da-kube-api-access-p6m8r\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583223    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-run-netns\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583253    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-867pr\" (UniqueName: \"kubernetes.io/projected/c8726c19-73c9-4166-97a2-88f09f2bb28c-kube-api-access-867pr\") pod \"node-ca-4v4mk\" (UID: \"c8726c19-73c9-4166-97a2-88f09f2bb28c\") " pod="openshift-image-registry/node-ca-4v4mk"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583298    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-run-k8s-cni-cncf-io\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583336    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e70ee258-310d-4f06-9908-afc7a27684da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583364    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwnj4\" (UniqueName: \"kubernetes.io/projected/53433330-3378-4829-be11-5a219e621ef4-kube-api-access-cwnj4\") pod \"node-resolver-pt6c7\" (UID: \"53433330-3378-4829-be11-5a219e621ef4\") " pod="openshift-dns/node-resolver-pt6c7"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583369    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53433330-3378-4829-be11-5a219e621ef4-tmp-dir\") pod \"node-resolver-pt6c7\" (UID: \"53433330-3378-4829-be11-5a219e621ef4\") " pod="openshift-dns/node-resolver-pt6c7"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583389    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-os-release\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583401    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c8726c19-73c9-4166-97a2-88f09f2bb28c-serviceca\") pod \"node-ca-4v4mk\" (UID: \"c8726c19-73c9-4166-97a2-88f09f2bb28c\") " pod="openshift-image-registry/node-ca-4v4mk"
Jul 13 21:29:20.584690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583426    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-var-lib-kubelet\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583448    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-os-release\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-run-multus-certs\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:20.583508    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583514    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-run-multus-certs\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583553    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f5a578cc-d952-452d-bc46-fa37ed934581-multus-daemon-config\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583560    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e70ee258-310d-4f06-9908-afc7a27684da-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583554    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-var-lib-kubelet\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:20.583578    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs podName:4b89f48b-c81d-40dc-a85d-ea77e98bc1f5 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:21.083558324 +0000 UTC m=+3.005607241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs") pod "network-metrics-daemon-fjzfw" (UID: "4b89f48b-c81d-40dc-a85d-ea77e98bc1f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583598    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e70ee258-310d-4f06-9908-afc7a27684da-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583609    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-host-run-netns\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5a578cc-d952-452d-bc46-fa37ed934581-cni-binary-copy\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583680    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-hostroot\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583740    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f5a578cc-d952-452d-bc46-fa37ed934581-hostroot\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.585154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.583990    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e70ee258-310d-4f06-9908-afc7a27684da-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.585592 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.584082    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e70ee258-310d-4f06-9908-afc7a27684da-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.585592 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.584114    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5a578cc-d952-452d-bc46-fa37ed934581-cni-binary-copy\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.589760 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:20.589697    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:29:20.589760 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:20.589722    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:29:20.589760 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:20.589735    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4p6xg for pod openshift-network-diagnostics/network-check-target-td5gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:20.590024 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:20.589799    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg podName:b25ee3f1-7546-49f6-bb7c-6f97b33574f8 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:21.089781022 +0000 UTC m=+3.011829931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4p6xg" (UniqueName: "kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg") pod "network-check-target-td5gj" (UID: "b25ee3f1-7546-49f6-bb7c-6f97b33574f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:20.592137 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.592108    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srpvm\" (UniqueName: \"kubernetes.io/projected/f5a578cc-d952-452d-bc46-fa37ed934581-kube-api-access-srpvm\") pod \"multus-nhhkd\" (UID: \"f5a578cc-d952-452d-bc46-fa37ed934581\") " pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.592476 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.592453    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-867pr\" (UniqueName: \"kubernetes.io/projected/c8726c19-73c9-4166-97a2-88f09f2bb28c-kube-api-access-867pr\") pod \"node-ca-4v4mk\" (UID: \"c8726c19-73c9-4166-97a2-88f09f2bb28c\") " pod="openshift-image-registry/node-ca-4v4mk"
Jul 13 21:29:20.592559 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.592461    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwnj4\" (UniqueName: \"kubernetes.io/projected/53433330-3378-4829-be11-5a219e621ef4-kube-api-access-cwnj4\") pod \"node-resolver-pt6c7\" (UID: \"53433330-3378-4829-be11-5a219e621ef4\") " pod="openshift-dns/node-resolver-pt6c7"
Jul 13 21:29:20.593185 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.593162    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6m8r\" (UniqueName: \"kubernetes.io/projected/e70ee258-310d-4f06-9908-afc7a27684da-kube-api-access-p6m8r\") pod \"multus-additional-cni-plugins-qlllp\" (UID: \"e70ee258-310d-4f06-9908-afc7a27684da\") " pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.593301 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.593245    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m7fh\" (UniqueName: \"kubernetes.io/projected/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-kube-api-access-8m7fh\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:20.649943 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.649913    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:29:20.661057 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.661018    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw"
Jul 13 21:29:20.668300 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.668274    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rrzxh"
Jul 13 21:29:20.679922 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.679896    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kt42z"
Jul 13 21:29:20.684147 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.684120    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:20.691936 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.691908    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9x42s"
Jul 13 21:29:20.699993 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.699807    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4v4mk"
Jul 13 21:29:20.706838 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.706628    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pt6c7"
Jul 13 21:29:20.714735 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.714569    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nhhkd"
Jul 13 21:29:20.720537 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.720367    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qlllp"
Jul 13 21:29:20.840523 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:20.840029    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8726c19_73c9_4166_97a2_88f09f2bb28c.slice/crio-aec4f7b75db96defb9a5ccc60c35a663ed28e93982f02e60497af1fa1b03c032 WatchSource:0}: Error finding container aec4f7b75db96defb9a5ccc60c35a663ed28e93982f02e60497af1fa1b03c032: Status 404 returned error can't find the container with id aec4f7b75db96defb9a5ccc60c35a663ed28e93982f02e60497af1fa1b03c032
Jul 13 21:29:20.867692 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:20.867659    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7af3cbf7_78c5_428d_a699_f0341cbd04d6.slice/crio-d949f4b4967b41f794fd771302a058b647ec7abd17011f7c251bf672dad724dc WatchSource:0}: Error finding container d949f4b4967b41f794fd771302a058b647ec7abd17011f7c251bf672dad724dc: Status 404 returned error can't find the container with id d949f4b4967b41f794fd771302a058b647ec7abd17011f7c251bf672dad724dc
Jul 13 21:29:20.868683 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:20.868633    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15165918_15d0_4f74_b300_931b0c8d67f5.slice/crio-3672198db671d4b057fbfe9d59d2681f029325bfbcaae591f3c94c20fecc7475 WatchSource:0}: Error finding container 3672198db671d4b057fbfe9d59d2681f029325bfbcaae591f3c94c20fecc7475: Status 404 returned error can't find the container with id 3672198db671d4b057fbfe9d59d2681f029325bfbcaae591f3c94c20fecc7475
Jul 13 21:29:20.872271 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:20.871637    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5a578cc_d952_452d_bc46_fa37ed934581.slice/crio-48aba3df1b489a86cf87b7fcc52801f668580d6115815b4dc37e9798f4c46d9c WatchSource:0}: Error finding container 48aba3df1b489a86cf87b7fcc52801f668580d6115815b4dc37e9798f4c46d9c: Status 404 returned error can't find the container with id 48aba3df1b489a86cf87b7fcc52801f668580d6115815b4dc37e9798f4c46d9c
Jul 13 21:29:20.872271 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:20.872060    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20215437_0510_495e_8a20_3a272973cdfd.slice/crio-8505a0d5763adb73337bc08e558fd97df68d36e1f3c3af46f38df7aeb31ab3b2 WatchSource:0}: Error finding container 8505a0d5763adb73337bc08e558fd97df68d36e1f3c3af46f38df7aeb31ab3b2: Status 404 returned error can't find the container with id 8505a0d5763adb73337bc08e558fd97df68d36e1f3c3af46f38df7aeb31ab3b2
Jul 13 21:29:20.987725 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:20.987692    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:20.987890 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:20.987872    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:20.987969 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:20.987956    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret podName:0cb23a38-df98-4abc-9ca3-43fe622a34c7 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:21.987933495 +0000 UTC m=+3.909982410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret") pod "global-pull-secret-syncer-nkmbp" (UID: "0cb23a38-df98-4abc-9ca3-43fe622a34c7") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:21.088609 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.088428    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:21.088770 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:21.088609    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:21.088770 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:21.088705    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs podName:4b89f48b-c81d-40dc-a85d-ea77e98bc1f5 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:22.088683838 +0000 UTC m=+4.010732734 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs") pod "network-metrics-daemon-fjzfw" (UID: "4b89f48b-c81d-40dc-a85d-ea77e98bc1f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:21.189068 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.189034    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p6xg\" (UniqueName: \"kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg\") pod \"network-check-target-td5gj\" (UID: \"b25ee3f1-7546-49f6-bb7c-6f97b33574f8\") " pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:21.189245 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:21.189208    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:29:21.189245 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:21.189225    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:29:21.189245 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:21.189235    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4p6xg for pod openshift-network-diagnostics/network-check-target-td5gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:21.189379 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:21.189292    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg podName:b25ee3f1-7546-49f6-bb7c-6f97b33574f8 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:22.189276396 +0000 UTC m=+4.111325307 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4p6xg" (UniqueName: "kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg") pod "network-check-target-td5gj" (UID: "b25ee3f1-7546-49f6-bb7c-6f97b33574f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:21.423520 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.423405    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:24:19 +0000 UTC" deadline="2028-02-13 18:24:30.947350078 +0000 UTC"
Jul 13 21:29:21.423520 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.423467    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13916h55m9.523886682s"
Jul 13 21:29:21.478025 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.477490    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal" event={"ID":"be4b5ee886bda5efae9a7f3c00ec6ced","Type":"ContainerStarted","Data":"95197766787ce8dffeb9c16adef967657b290d88c75e0a602451c7632a5b4f61"}
Jul 13 21:29:21.479195 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.479158    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb" event={"ID":"15165918-15d0-4f74-b300-931b0c8d67f5","Type":"ContainerStarted","Data":"3672198db671d4b057fbfe9d59d2681f029325bfbcaae591f3c94c20fecc7475"}
Jul 13 21:29:21.480445 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.480422    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rrzxh" event={"ID":"abaedb55-25f3-46db-a793-cc0a4f71ddd3","Type":"ContainerStarted","Data":"99da55f089ee43dfb2e680992a788203dd660947c5574ce1813da612ae6a242e"}
Jul 13 21:29:21.481644 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.481622    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9x42s" event={"ID":"20215437-0510-495e-8a20-3a272973cdfd","Type":"ContainerStarted","Data":"8505a0d5763adb73337bc08e558fd97df68d36e1f3c3af46f38df7aeb31ab3b2"}
Jul 13 21:29:21.482777 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.482756    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw" event={"ID":"7af3cbf7-78c5-428d-a699-f0341cbd04d6","Type":"ContainerStarted","Data":"d949f4b4967b41f794fd771302a058b647ec7abd17011f7c251bf672dad724dc"}
Jul 13 21:29:21.483768 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.483747    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kt42z" event={"ID":"fd9888e0-94ba-4da2-b458-8bad7d8a136b","Type":"ContainerStarted","Data":"9ab5c092845ab400433f4dffc55b5610adb1fe50db1051f97d343ac64100b57e"}
Jul 13 21:29:21.484745 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.484725    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlllp" event={"ID":"e70ee258-310d-4f06-9908-afc7a27684da","Type":"ContainerStarted","Data":"394034b98167e5f15c6673f5ae246d6b4271fcb11749aa7b2b45452a52800972"}
Jul 13 21:29:21.485739 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.485719    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pt6c7" event={"ID":"53433330-3378-4829-be11-5a219e621ef4","Type":"ContainerStarted","Data":"35be547900b6d874cf44601c5623ee75fefd86e26d9db15688fa7344359efef2"}
Jul 13 21:29:21.487209 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.487189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nhhkd" event={"ID":"f5a578cc-d952-452d-bc46-fa37ed934581","Type":"ContainerStarted","Data":"48aba3df1b489a86cf87b7fcc52801f668580d6115815b4dc37e9798f4c46d9c"}
Jul 13 21:29:21.488168 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.488148    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4v4mk" event={"ID":"c8726c19-73c9-4166-97a2-88f09f2bb28c","Type":"ContainerStarted","Data":"aec4f7b75db96defb9a5ccc60c35a663ed28e93982f02e60497af1fa1b03c032"}
Jul 13 21:29:21.996145 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:21.995817    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:21.996145 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:21.995995    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:21.996145 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:21.996059    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret podName:0cb23a38-df98-4abc-9ca3-43fe622a34c7 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:23.996041242 +0000 UTC m=+5.918090153 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret") pod "global-pull-secret-syncer-nkmbp" (UID: "0cb23a38-df98-4abc-9ca3-43fe622a34c7") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:22.019381 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:22.019351    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:29:22.096977 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:22.096928    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:22.097246 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:22.097227    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:22.097339 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:22.097296    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs podName:4b89f48b-c81d-40dc-a85d-ea77e98bc1f5 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:24.097277108 +0000 UTC m=+6.019326021 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs") pod "network-metrics-daemon-fjzfw" (UID: "4b89f48b-c81d-40dc-a85d-ea77e98bc1f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:22.198465 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:22.198424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p6xg\" (UniqueName: \"kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg\") pod \"network-check-target-td5gj\" (UID: \"b25ee3f1-7546-49f6-bb7c-6f97b33574f8\") " pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:22.198752 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:22.198625    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:29:22.198752 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:22.198660    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:29:22.198752 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:22.198675    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4p6xg for pod openshift-network-diagnostics/network-check-target-td5gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:22.198752 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:22.198736    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg podName:b25ee3f1-7546-49f6-bb7c-6f97b33574f8 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:24.198715386 +0000 UTC m=+6.120764284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4p6xg" (UniqueName: "kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg") pod "network-check-target-td5gj" (UID: "b25ee3f1-7546-49f6-bb7c-6f97b33574f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:22.474401 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:22.474332    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:22.474554 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:22.474458    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:22.474883 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:22.474862    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:22.474977 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:22.474957    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:22.475038 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:22.475015    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:22.475113 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:22.475094    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:22.511441 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:22.511256    2564 generic.go:356] "Generic (PLEG): container finished" podID="be4b5ee886bda5efae9a7f3c00ec6ced" containerID="95197766787ce8dffeb9c16adef967657b290d88c75e0a602451c7632a5b4f61" exitCode=0
Jul 13 21:29:22.511441 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:22.511358    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal" event={"ID":"be4b5ee886bda5efae9a7f3c00ec6ced","Type":"ContainerDied","Data":"95197766787ce8dffeb9c16adef967657b290d88c75e0a602451c7632a5b4f61"}
Jul 13 21:29:22.532918 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:22.532808    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-98.ec2.internal" event={"ID":"09e44c48acdcebe81c75239f1dc1647b","Type":"ContainerStarted","Data":"0a3d986dc57a305c1617a1bb65a95b95e5fe6868aded4f3f7b5a545f678e7a77"}
Jul 13 21:29:23.563501 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:23.563427    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal" event={"ID":"be4b5ee886bda5efae9a7f3c00ec6ced","Type":"ContainerStarted","Data":"1b74f2db4316a59854682a6684dd1a5e12dd0481c92afa5c9fe97c305a380e6b"}
Jul 13 21:29:23.579705 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:23.578487    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-98.ec2.internal" podStartSLOduration=4.578464103 podStartE2EDuration="4.578464103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:22.546317733 +0000 UTC m=+4.468366636" watchObservedRunningTime="2026-07-13 21:29:23.578464103 +0000 UTC m=+5.500513022"
Jul 13 21:29:24.021271 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:24.021152    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:24.021433 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:24.021375    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:24.021489 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:24.021440    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret podName:0cb23a38-df98-4abc-9ca3-43fe622a34c7 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:28.021420109 +0000 UTC m=+9.943469016 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret") pod "global-pull-secret-syncer-nkmbp" (UID: "0cb23a38-df98-4abc-9ca3-43fe622a34c7") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:24.122546 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:24.121972    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:24.122546 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:24.122118    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:24.122546 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:24.122183    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs podName:4b89f48b-c81d-40dc-a85d-ea77e98bc1f5 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:28.122164091 +0000 UTC m=+10.044213008 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs") pod "network-metrics-daemon-fjzfw" (UID: "4b89f48b-c81d-40dc-a85d-ea77e98bc1f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:24.222903 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:24.222553    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p6xg\" (UniqueName: \"kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg\") pod \"network-check-target-td5gj\" (UID: \"b25ee3f1-7546-49f6-bb7c-6f97b33574f8\") " pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:24.222903 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:24.222730    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:29:24.222903 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:24.222756    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:29:24.222903 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:24.222772    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4p6xg for pod openshift-network-diagnostics/network-check-target-td5gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:24.222903 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:24.222838    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg podName:b25ee3f1-7546-49f6-bb7c-6f97b33574f8 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:28.22282074 +0000 UTC m=+10.144869639 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4p6xg" (UniqueName: "kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg") pod "network-check-target-td5gj" (UID: "b25ee3f1-7546-49f6-bb7c-6f97b33574f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:24.467386 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:24.467300    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:24.467546 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:24.467451    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:24.470385 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:24.467733    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:24.470385 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:24.467830    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:24.470385 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:24.467881    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:24.470385 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:24.467962    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:26.473142 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:26.472335    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:26.473142 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:26.472464    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:26.473142 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:26.472879    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:26.473142 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:26.472966    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:26.473142 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:26.473026    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:26.473142 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:26.473101    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:28.055365 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:28.055318    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:28.055864 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:28.055493    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:28.055864 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:28.055570    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret podName:0cb23a38-df98-4abc-9ca3-43fe622a34c7 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:36.055552347 +0000 UTC m=+17.977601245 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret") pod "global-pull-secret-syncer-nkmbp" (UID: "0cb23a38-df98-4abc-9ca3-43fe622a34c7") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:28.155844 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:28.155717    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:28.156036 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:28.155882    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:28.156036 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:28.155953    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs podName:4b89f48b-c81d-40dc-a85d-ea77e98bc1f5 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:36.155934558 +0000 UTC m=+18.077983465 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs") pod "network-metrics-daemon-fjzfw" (UID: "4b89f48b-c81d-40dc-a85d-ea77e98bc1f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:28.256396 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:28.256355    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p6xg\" (UniqueName: \"kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg\") pod \"network-check-target-td5gj\" (UID: \"b25ee3f1-7546-49f6-bb7c-6f97b33574f8\") " pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:28.256564 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:28.256525    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:29:28.256564 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:28.256543    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:29:28.256564 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:28.256552    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4p6xg for pod openshift-network-diagnostics/network-check-target-td5gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:28.256761 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:28.256606    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg podName:b25ee3f1-7546-49f6-bb7c-6f97b33574f8 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:36.256590405 +0000 UTC m=+18.178639320 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4p6xg" (UniqueName: "kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg") pod "network-check-target-td5gj" (UID: "b25ee3f1-7546-49f6-bb7c-6f97b33574f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:28.469156 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:28.469078    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:28.469311 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:28.469191    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:28.469311 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:28.469209    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:28.469311 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:28.469245    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:28.469466 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:28.469339    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:28.469466 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:28.469411    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:30.471337 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:30.470864    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:30.471337 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:30.470980    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:30.471337 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:30.470990    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:30.471337 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:30.471012    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:30.471337 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:30.471093    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:30.471337 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:30.471162    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:32.470305 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:32.470271    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:32.470732 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:32.470323    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:32.470732 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:32.470271    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:32.470732 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:32.470384    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:32.470732 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:32.470460    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:32.470732 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:32.470543    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:34.467029 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:34.466985    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:34.467478 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:34.467000    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:34.467478 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:34.467158    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:34.467478 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:34.467151    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:34.467478 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:34.467282    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:34.467478 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:34.467430    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:36.122936 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:36.122906    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:36.123331 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:36.123048    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:36.123331 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:36.123112    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret podName:0cb23a38-df98-4abc-9ca3-43fe622a34c7 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:52.123096075 +0000 UTC m=+34.045144970 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret") pod "global-pull-secret-syncer-nkmbp" (UID: "0cb23a38-df98-4abc-9ca3-43fe622a34c7") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:36.223397 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:36.223361    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:36.223507 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:36.223477    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:36.223539 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:36.223528    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs podName:4b89f48b-c81d-40dc-a85d-ea77e98bc1f5 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:52.223513839 +0000 UTC m=+34.145562734 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs") pod "network-metrics-daemon-fjzfw" (UID: "4b89f48b-c81d-40dc-a85d-ea77e98bc1f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:36.324608 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:36.324580    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p6xg\" (UniqueName: \"kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg\") pod \"network-check-target-td5gj\" (UID: \"b25ee3f1-7546-49f6-bb7c-6f97b33574f8\") " pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:36.324768 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:36.324750    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:29:36.324803 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:36.324775    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:29:36.324803 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:36.324784    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4p6xg for pod openshift-network-diagnostics/network-check-target-td5gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:36.324872 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:36.324833    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg podName:b25ee3f1-7546-49f6-bb7c-6f97b33574f8 nodeName:}" failed. No retries permitted until 2026-07-13 21:29:52.324818845 +0000 UTC m=+34.246867741 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4p6xg" (UniqueName: "kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg") pod "network-check-target-td5gj" (UID: "b25ee3f1-7546-49f6-bb7c-6f97b33574f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:36.467637 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:36.467493    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:36.467637 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:36.467543    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:36.467637 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:36.467504    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:36.467943 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:36.467642    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:36.467943 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:36.467736    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:36.467943 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:36.467811    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:38.470269 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:38.470233    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:38.470269 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:38.470270    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:38.470782 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:38.470233    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:38.470782 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:38.470396    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:38.470782 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:38.470475    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:38.470782 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:38.470568    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:40.467521 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:40.467473    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:40.468091 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:40.467483    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:40.468091 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:40.467593    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:40.468091 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:40.467605    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:40.468091 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:40.467751    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:40.468091 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:40.467870    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:41.622682 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.622355    2564 generic.go:356] "Generic (PLEG): container finished" podID="e70ee258-310d-4f06-9908-afc7a27684da" containerID="d4964eb1e0ef90d327651a2a29aa400a3be9c32f7fd4057c9870670779461300" exitCode=0
Jul 13 21:29:41.623524 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.622444    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlllp" event={"ID":"e70ee258-310d-4f06-9908-afc7a27684da","Type":"ContainerDied","Data":"d4964eb1e0ef90d327651a2a29aa400a3be9c32f7fd4057c9870670779461300"}
Jul 13 21:29:41.624282 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.624172    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pt6c7" event={"ID":"53433330-3378-4829-be11-5a219e621ef4","Type":"ContainerStarted","Data":"3f022f628b1ae5b13bfc2843f7b3966eb359225f7e531aa7c66c5b0e44b601ea"}
Jul 13 21:29:41.625694 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.625672    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nhhkd" event={"ID":"f5a578cc-d952-452d-bc46-fa37ed934581","Type":"ContainerStarted","Data":"c0e77af302d47576e8f00e96cdd507e77816b91c3770da9990a4df88589a8f71"}
Jul 13 21:29:41.627117 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.627090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4v4mk" event={"ID":"c8726c19-73c9-4166-97a2-88f09f2bb28c","Type":"ContainerStarted","Data":"fe055381583ae4c720b540f07df47b33e2d76db70c115164123b1a6d4d268919"}
Jul 13 21:29:41.630165 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.630148    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:29:41.630452 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.630433    2564 generic.go:356] "Generic (PLEG): container finished" podID="15165918-15d0-4f74-b300-931b0c8d67f5" containerID="c529cb8b43ed5410ae3a66efbaeac6de9bfca0d0bb30262a2d52567a19718c20" exitCode=1
Jul 13 21:29:41.630535 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.630498    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb" event={"ID":"15165918-15d0-4f74-b300-931b0c8d67f5","Type":"ContainerStarted","Data":"14f68a7057eccc951a0543ac64b491a7ab8a05d1f3e8275b919428a797b1f6ed"}
Jul 13 21:29:41.630590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.630554    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb" event={"ID":"15165918-15d0-4f74-b300-931b0c8d67f5","Type":"ContainerStarted","Data":"452b6a3f76c566553b4bcc1bdc303d3fdeddbefde746faac65ca67ede0f2c102"}
Jul 13 21:29:41.630632 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.630600    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb" event={"ID":"15165918-15d0-4f74-b300-931b0c8d67f5","Type":"ContainerStarted","Data":"4535dc17dcd617dd69f6d3ca9d9b1253647e9ac38585963ac2f97f31f3963b3d"}
Jul 13 21:29:41.630697 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.630635    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb" event={"ID":"15165918-15d0-4f74-b300-931b0c8d67f5","Type":"ContainerStarted","Data":"bb63ad77c1697287d790d97cf11ddd558cc6711d14837028c449f02da79e3c07"}
Jul 13 21:29:41.630697 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.630689    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb" event={"ID":"15165918-15d0-4f74-b300-931b0c8d67f5","Type":"ContainerDied","Data":"c529cb8b43ed5410ae3a66efbaeac6de9bfca0d0bb30262a2d52567a19718c20"}
Jul 13 21:29:41.630768 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.630733    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb" event={"ID":"15165918-15d0-4f74-b300-931b0c8d67f5","Type":"ContainerStarted","Data":"fa5e9c1f5f67589572f9a0edd88f191c7e33715b5d2d93dbed524d610f6d1bd6"}
Jul 13 21:29:41.631709 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.631692    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rrzxh" event={"ID":"abaedb55-25f3-46db-a793-cc0a4f71ddd3","Type":"ContainerStarted","Data":"32cc6cab6be1aefe82db75daca353efbdae8b5f84e949bc8d844533b85a8f556"}
Jul 13 21:29:41.633010 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.632992    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9x42s" event={"ID":"20215437-0510-495e-8a20-3a272973cdfd","Type":"ContainerStarted","Data":"37f19cb876b8c2d6655a74599d7bb30906434720c111d095a9bcbf0242d7b74e"}
Jul 13 21:29:41.634257 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.634239    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw" event={"ID":"7af3cbf7-78c5-428d-a699-f0341cbd04d6","Type":"ContainerStarted","Data":"bbf7e83c2ff8e27fe5b6f7df29f10f5329002f2b0edec09bc23d273f95cb0380"}
Jul 13 21:29:41.644112 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.644074    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-98.ec2.internal" podStartSLOduration=22.644062162 podStartE2EDuration="22.644062162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:23.579686785 +0000 UTC m=+5.501735703" watchObservedRunningTime="2026-07-13 21:29:41.644062162 +0000 UTC m=+23.566111079"
Jul 13 21:29:41.661269 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.661225    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pt6c7" podStartSLOduration=4.029961116 podStartE2EDuration="23.661210744s" totalImagesPullingTime="19.631249628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:41.660977971 +0000 UTC m=+23.583026879" watchObservedRunningTime="2026-07-13 21:29:41.661210744 +0000 UTC m=+23.583259661"
Jul 13 21:29:41.675367 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.675318    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rrzxh" podStartSLOduration=4.052473268 podStartE2EDuration="23.675303404s" totalImagesPullingTime="19.622830136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:41.674753332 +0000 UTC m=+23.596802248" watchObservedRunningTime="2026-07-13 21:29:41.675303404 +0000 UTC m=+23.597352320"
Jul 13 21:29:41.694208 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.694170    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9x42s" podStartSLOduration=4.070171343 podStartE2EDuration="23.694155144s" totalImagesPullingTime="19.623983801s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:41.694139546 +0000 UTC m=+23.616188463" watchObservedRunningTime="2026-07-13 21:29:41.694155144 +0000 UTC m=+23.616204061"
Jul 13 21:29:41.709699 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.709623    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4v4mk" podStartSLOduration=4.064107907 podStartE2EDuration="23.709608753s" totalImagesPullingTime="19.645500846s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:41.709196321 +0000 UTC m=+23.631245238" watchObservedRunningTime="2026-07-13 21:29:41.709608753 +0000 UTC m=+23.631657670"
Jul 13 21:29:41.725318 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.725272    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nhhkd" podStartSLOduration=4.059662408 podStartE2EDuration="23.725256201s" totalImagesPullingTime="19.665593793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:41.725039017 +0000 UTC m=+23.647087934" watchObservedRunningTime="2026-07-13 21:29:41.725256201 +0000 UTC m=+23.647305117"
Jul 13 21:29:41.978344 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.978313    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rrzxh"
Jul 13 21:29:41.978872 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:41.978854    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rrzxh"
Jul 13 21:29:42.090052 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:42.090026    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 13 21:29:42.445294 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:42.445098    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T21:29:42.090045607Z","UUID":"fbe4a0ed-4910-4eb6-a9e0-cea2a987b243","Handler":null,"Name":"","Endpoint":""}
Jul 13 21:29:42.446641 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:42.446613    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 13 21:29:42.446804 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:42.446667    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 13 21:29:42.470891 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:42.470831    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:42.471070 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:42.470869    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:42.471070 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:42.470876    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:42.471070 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:42.471013    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:42.471208 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:42.471067    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:42.471208 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:42.471144    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:42.638958 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:42.638925    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw" event={"ID":"7af3cbf7-78c5-428d-a699-f0341cbd04d6","Type":"ContainerStarted","Data":"bf9c0c1727ef9f6bb03289eade602fc4c7b076770f302c50586955291d1cb1e0"}
Jul 13 21:29:42.640703 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:42.640637    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kt42z" event={"ID":"fd9888e0-94ba-4da2-b458-8bad7d8a136b","Type":"ContainerStarted","Data":"cf904b9458b086352501bceee5b6c6b483207be457eca3a0cdebf5fcbc4f3442"}
Jul 13 21:29:42.654689 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:42.654619    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kt42z" podStartSLOduration=5.035845794 podStartE2EDuration="24.654600415s" totalImagesPullingTime="19.618754621s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:42.654059249 +0000 UTC m=+24.576108168" watchObservedRunningTime="2026-07-13 21:29:42.654600415 +0000 UTC m=+24.576649337"
Jul 13 21:29:43.646889 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:43.646577    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:29:43.647457 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:43.647357    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb" event={"ID":"15165918-15d0-4f74-b300-931b0c8d67f5","Type":"ContainerStarted","Data":"d801ba6ca9a993bcea631a7bca0b718be1f07c824437652a8397aae69d760ea3"}
Jul 13 21:29:43.650012 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:43.649983    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw" event={"ID":"7af3cbf7-78c5-428d-a699-f0341cbd04d6","Type":"ContainerStarted","Data":"802188b38317ab8f5211ad23c613e4bc05ccd66f87236203dfa577a2408266dd"}
Jul 13 21:29:43.650142 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:43.650058    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 13 21:29:43.667635 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:43.667567    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-glccw" podStartSLOduration=3.567210009 podStartE2EDuration="25.667551964s" totalImagesPullingTime="22.100341955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:43.666584647 +0000 UTC m=+25.588633564" watchObservedRunningTime="2026-07-13 21:29:43.667551964 +0000 UTC m=+25.589600882"
Jul 13 21:29:44.466864 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:44.466834    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:44.467168 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:44.466834    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:44.467168 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:44.466949    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:44.467168 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:44.467034    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:44.467168 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:44.466842    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:44.467168 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:44.467145    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:44.578365 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:44.578327    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rrzxh"
Jul 13 21:29:44.579020 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:44.578998    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rrzxh"
Jul 13 21:29:46.467931 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:46.467682    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:46.468622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:46.467682    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:46.468622 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:46.467958    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:46.468622 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:46.468056    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:46.468622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:46.467736    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:46.468622 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:46.468138    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:46.659597 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:46.659560    2564 generic.go:356] "Generic (PLEG): container finished" podID="e70ee258-310d-4f06-9908-afc7a27684da" containerID="e7ee83168d79e2b8f09f921321c458eca3b910790eb90ee36c44a2a0aa1750d8" exitCode=0
Jul 13 21:29:46.659782 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:46.659656    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlllp" event={"ID":"e70ee258-310d-4f06-9908-afc7a27684da","Type":"ContainerDied","Data":"e7ee83168d79e2b8f09f921321c458eca3b910790eb90ee36c44a2a0aa1750d8"}
Jul 13 21:29:46.663169 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:46.663151    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:29:46.663493 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:46.663475    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb" event={"ID":"15165918-15d0-4f74-b300-931b0c8d67f5","Type":"ContainerStarted","Data":"ff8e9ffb29916f7f829cb9cb70bf742b72dd56ff160f4cd43a08b7af20e2dd76"}
Jul 13 21:29:46.663704 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:46.663690    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:46.663770 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:46.663740    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:46.663974 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:46.663957    2564 scope.go:117] "RemoveContainer" containerID="c529cb8b43ed5410ae3a66efbaeac6de9bfca0d0bb30262a2d52567a19718c20"
Jul 13 21:29:46.678998 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:46.678980    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:47.671483 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:47.671454    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:29:47.672090 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:47.671869    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb" event={"ID":"15165918-15d0-4f74-b300-931b0c8d67f5","Type":"ContainerStarted","Data":"66c59ccf8441c96fc1172998c29116dc44aa5181b1e78d169763d1150b0cc149"}
Jul 13 21:29:47.672416 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:47.672290    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:47.689573 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:47.689543    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:29:47.700706 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:47.700632    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb" podStartSLOduration=9.96234415 podStartE2EDuration="29.700611589s" totalImagesPullingTime="19.738267439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:47.700533019 +0000 UTC m=+29.622581960" watchObservedRunningTime="2026-07-13 21:29:47.700611589 +0000 UTC m=+29.622660508"
Jul 13 21:29:47.926085 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:47.925855    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-td5gj"]
Jul 13 21:29:47.926235 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:47.926145    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:47.926281 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:47.926234    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:47.926561 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:47.926543    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fjzfw"]
Jul 13 21:29:47.926643 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:47.926633    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:47.926743 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:47.926726    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:47.936443 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:47.936420    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nkmbp"]
Jul 13 21:29:47.936544 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:47.936497    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:47.936594 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:47.936581    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:48.676489 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:48.676457    2564 generic.go:356] "Generic (PLEG): container finished" podID="e70ee258-310d-4f06-9908-afc7a27684da" containerID="efe571c9b94bc1c54ffbdc030deede3e9402a4cf7cf7252346ec7084808231d4" exitCode=0
Jul 13 21:29:48.677049 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:48.676544    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlllp" event={"ID":"e70ee258-310d-4f06-9908-afc7a27684da","Type":"ContainerDied","Data":"efe571c9b94bc1c54ffbdc030deede3e9402a4cf7cf7252346ec7084808231d4"}
Jul 13 21:29:49.467267 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:49.467229    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:49.467447 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:49.467233    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:49.467447 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:49.467370    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:49.467558 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:49.467455    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:49.467558 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:49.467243    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:49.467558 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:49.467551    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:50.682104 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:50.682073    2564 generic.go:356] "Generic (PLEG): container finished" podID="e70ee258-310d-4f06-9908-afc7a27684da" containerID="1ea824d395cc09b1e2596fbbb7b17b192cbe971d469fd41052dec6121487d88b" exitCode=0
Jul 13 21:29:50.682626 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:50.682114    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlllp" event={"ID":"e70ee258-310d-4f06-9908-afc7a27684da","Type":"ContainerDied","Data":"1ea824d395cc09b1e2596fbbb7b17b192cbe971d469fd41052dec6121487d88b"}
Jul 13 21:29:51.467116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:51.467077    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:51.467116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:51.467114    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:51.467360 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:51.467079    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:51.467360 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:51.467195    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkmbp" podUID="0cb23a38-df98-4abc-9ca3-43fe622a34c7"
Jul 13 21:29:51.467360 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:51.467317    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:29:51.467514 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:51.467398    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-td5gj" podUID="b25ee3f1-7546-49f6-bb7c-6f97b33574f8"
Jul 13 21:29:52.136516 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:52.136474    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:52.136959 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:52.136610    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:52.136959 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:52.136696    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret podName:0cb23a38-df98-4abc-9ca3-43fe622a34c7 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:24.136677681 +0000 UTC m=+66.058726577 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret") pod "global-pull-secret-syncer-nkmbp" (UID: "0cb23a38-df98-4abc-9ca3-43fe622a34c7") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:29:52.237541 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:52.237504    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:52.237732 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:52.237686    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:52.237791 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:52.237762    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs podName:4b89f48b-c81d-40dc-a85d-ea77e98bc1f5 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:24.237741397 +0000 UTC m=+66.159790293 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs") pod "network-metrics-daemon-fjzfw" (UID: "4b89f48b-c81d-40dc-a85d-ea77e98bc1f5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:29:52.338020 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:52.337976    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p6xg\" (UniqueName: \"kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg\") pod \"network-check-target-td5gj\" (UID: \"b25ee3f1-7546-49f6-bb7c-6f97b33574f8\") " pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:52.338206 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:52.338168    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:29:52.338206 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:52.338194    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:29:52.338206 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:52.338208    2564 projected.go:196] Error preparing data for projected volume kube-api-access-4p6xg for pod openshift-network-diagnostics/network-check-target-td5gj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:52.338369 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:52.338266    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg podName:b25ee3f1-7546-49f6-bb7c-6f97b33574f8 nodeName:}" failed. No retries permitted until 2026-07-13 21:30:24.33824798 +0000 UTC m=+66.260296875 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4p6xg" (UniqueName: "kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg") pod "network-check-target-td5gj" (UID: "b25ee3f1-7546-49f6-bb7c-6f97b33574f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:29:53.419999 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.419659    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-98.ec2.internal" event="NodeReady"
Jul 13 21:29:53.420419 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.420053    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 13 21:29:53.451522 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.451494    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79f645bcc9-drsxw"]
Jul 13 21:29:53.456661 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.456618    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"]
Jul 13 21:29:53.456808 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.456735    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.459184 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.459146    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 13 21:29:53.459184 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.459151    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 13 21:29:53.459363 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.459208    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 13 21:29:53.459363 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.459331    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rpb8r\""
Jul 13 21:29:53.460154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.460132    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:29:53.462551 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.462531    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-khk5w\""
Jul 13 21:29:53.462707 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.462577    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 13 21:29:53.463280 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.463258    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 13 21:29:53.466224 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.466201    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 13 21:29:53.466755 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.466694    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:29:53.467193 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.467175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:29:53.468096 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.467599    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:29:53.468632 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.468614    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"]
Jul 13 21:29:53.469231 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.469211    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79f645bcc9-drsxw"]
Jul 13 21:29:53.471059 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.470998    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 21:29:53.471365 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.471345    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 21:29:53.471608 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.471591    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 21:29:53.471800 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.471785    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 21:29:53.472003 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.471985    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-89gzj\""
Jul 13 21:29:53.472003 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.472000    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qsrtw\""
Jul 13 21:29:53.472816 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.472691    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xcd5s"]
Jul 13 21:29:53.476775 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.476757    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n2ltx"]
Jul 13 21:29:53.476940 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.476880    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:53.480333 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.480306    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n2ltx"
Jul 13 21:29:53.481672 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.480492    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 13 21:29:53.481672 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.480605    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l6v89\""
Jul 13 21:29:53.481672 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.480987    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 13 21:29:53.483569 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.483547    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 13 21:29:53.483569 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.483566    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 13 21:29:53.483852 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.483836    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wlljv\""
Jul 13 21:29:53.483967 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.483950    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 13 21:29:53.489402 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.489380    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n2ltx"]
Jul 13 21:29:53.489505 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.489427    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xcd5s"]
Jul 13 21:29:53.546750 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.546719    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/860f54f5-378e-478c-a414-dc546c02c05b-trusted-ca\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.546929 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.546766    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/860f54f5-378e-478c-a414-dc546c02c05b-image-registry-private-configuration\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.546929 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.546796    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.546929 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.546822    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/860f54f5-378e-478c-a414-dc546c02c05b-ca-trust-extracted\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.546929 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.546854    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr8x6\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-kube-api-access-hr8x6\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.546929 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.546907    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8cc99873-ff7d-4ebe-80d8-3147d30440ca-nginx-conf\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:29:53.547128 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.546948    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/860f54f5-378e-478c-a414-dc546c02c05b-installation-pull-secrets\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.547128 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.547022    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:29:53.547128 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.547050    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-bound-sa-token\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.547128 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.547070    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/860f54f5-378e-478c-a414-dc546c02c05b-registry-certificates\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.647563 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.647517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/860f54f5-378e-478c-a414-dc546c02c05b-trusted-ca\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.647757 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.647575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/860f54f5-378e-478c-a414-dc546c02c05b-image-registry-private-configuration\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.647757 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.647732    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d6hm\" (UniqueName: \"kubernetes.io/projected/4418e5fc-2e14-430d-aee2-37c5eeae27bb-kube-api-access-2d6hm\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:53.647881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.647783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.647881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.647810    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/860f54f5-378e-478c-a414-dc546c02c05b-ca-trust-extracted\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.647881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.647865    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr8x6\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-kube-api-access-hr8x6\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.648005 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.647898    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8cc99873-ff7d-4ebe-80d8-3147d30440ca-nginx-conf\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:29:53.648005 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.647921    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/860f54f5-378e-478c-a414-dc546c02c05b-installation-pull-secrets\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.648005 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.647945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:53.648005 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:53.647952    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:29:53.648005 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:53.647969    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f645bcc9-drsxw: secret "image-registry-tls" not found
Jul 13 21:29:53.648005 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.647982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4418e5fc-2e14-430d-aee2-37c5eeae27bb-tmp-dir\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:53.648308 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:53.648024    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls podName:860f54f5-378e-478c-a414-dc546c02c05b nodeName:}" failed. No retries permitted until 2026-07-13 21:29:54.148003601 +0000 UTC m=+36.070052510 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls") pod "image-registry-79f645bcc9-drsxw" (UID: "860f54f5-378e-478c-a414-dc546c02c05b") : secret "image-registry-tls" not found
Jul 13 21:29:53.648308 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.648281    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/860f54f5-378e-478c-a414-dc546c02c05b-ca-trust-extracted\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.648428 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.648329    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:29:53.648428 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.648368    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-bound-sa-token\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.648428 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.648396    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clsw9\" (UniqueName: \"kubernetes.io/projected/fc4ac07c-35b4-46be-8cda-9c35d1fc97bd-kube-api-access-clsw9\") pod \"ingress-canary-n2ltx\" (UID: \"fc4ac07c-35b4-46be-8cda-9c35d1fc97bd\") " pod="openshift-ingress-canary/ingress-canary-n2ltx"
Jul 13 21:29:53.648532 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.648431    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/860f54f5-378e-478c-a414-dc546c02c05b-registry-certificates\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.648532 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.648458    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4418e5fc-2e14-430d-aee2-37c5eeae27bb-config-volume\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:53.648532 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.648491    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc4ac07c-35b4-46be-8cda-9c35d1fc97bd-cert\") pod \"ingress-canary-n2ltx\" (UID: \"fc4ac07c-35b4-46be-8cda-9c35d1fc97bd\") " pod="openshift-ingress-canary/ingress-canary-n2ltx"
Jul 13 21:29:53.648665 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.648630    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/860f54f5-378e-478c-a414-dc546c02c05b-trusted-ca\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.649257 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:53.648846    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 21:29:53.649257 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:53.649213    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert podName:8cc99873-ff7d-4ebe-80d8-3147d30440ca nodeName:}" failed. No retries permitted until 2026-07-13 21:29:54.149196555 +0000 UTC m=+36.071245465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6dsxq" (UID: "8cc99873-ff7d-4ebe-80d8-3147d30440ca") : secret "networking-console-plugin-cert" not found
Jul 13 21:29:53.649257 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.649115    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/860f54f5-378e-478c-a414-dc546c02c05b-registry-certificates\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.649257 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.648940    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8cc99873-ff7d-4ebe-80d8-3147d30440ca-nginx-conf\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:29:53.652641 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.652583    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/860f54f5-378e-478c-a414-dc546c02c05b-installation-pull-secrets\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.652641 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.652585    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/860f54f5-378e-478c-a414-dc546c02c05b-image-registry-private-configuration\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.656326 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.656301    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-bound-sa-token\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.657042 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.657019    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr8x6\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-kube-api-access-hr8x6\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:53.749033 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.748945    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d6hm\" (UniqueName: \"kubernetes.io/projected/4418e5fc-2e14-430d-aee2-37c5eeae27bb-kube-api-access-2d6hm\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:53.749197 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.749043    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:53.749197 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.749084    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4418e5fc-2e14-430d-aee2-37c5eeae27bb-tmp-dir\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:53.749197 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.749119    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clsw9\" (UniqueName: \"kubernetes.io/projected/fc4ac07c-35b4-46be-8cda-9c35d1fc97bd-kube-api-access-clsw9\") pod \"ingress-canary-n2ltx\" (UID: \"fc4ac07c-35b4-46be-8cda-9c35d1fc97bd\") " pod="openshift-ingress-canary/ingress-canary-n2ltx"
Jul 13 21:29:53.749197 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.749136    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4418e5fc-2e14-430d-aee2-37c5eeae27bb-config-volume\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:53.749197 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.749155    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc4ac07c-35b4-46be-8cda-9c35d1fc97bd-cert\") pod \"ingress-canary-n2ltx\" (UID: \"fc4ac07c-35b4-46be-8cda-9c35d1fc97bd\") " pod="openshift-ingress-canary/ingress-canary-n2ltx"
Jul 13 21:29:53.749197 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:53.749156    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:29:53.749483 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:53.749230    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls podName:4418e5fc-2e14-430d-aee2-37c5eeae27bb nodeName:}" failed. No retries permitted until 2026-07-13 21:29:54.249211526 +0000 UTC m=+36.171260435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls") pod "dns-default-xcd5s" (UID: "4418e5fc-2e14-430d-aee2-37c5eeae27bb") : secret "dns-default-metrics-tls" not found
Jul 13 21:29:53.749526 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.749512    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4418e5fc-2e14-430d-aee2-37c5eeae27bb-tmp-dir\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:53.749863 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.749826    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4418e5fc-2e14-430d-aee2-37c5eeae27bb-config-volume\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:53.752107 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.752083    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc4ac07c-35b4-46be-8cda-9c35d1fc97bd-cert\") pod \"ingress-canary-n2ltx\" (UID: \"fc4ac07c-35b4-46be-8cda-9c35d1fc97bd\") " pod="openshift-ingress-canary/ingress-canary-n2ltx"
Jul 13 21:29:53.765774 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.765741    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d6hm\" (UniqueName: \"kubernetes.io/projected/4418e5fc-2e14-430d-aee2-37c5eeae27bb-kube-api-access-2d6hm\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:53.766716 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.766690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clsw9\" (UniqueName: \"kubernetes.io/projected/fc4ac07c-35b4-46be-8cda-9c35d1fc97bd-kube-api-access-clsw9\") pod \"ingress-canary-n2ltx\" (UID: \"fc4ac07c-35b4-46be-8cda-9c35d1fc97bd\") " pod="openshift-ingress-canary/ingress-canary-n2ltx"
Jul 13 21:29:53.819292 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:53.819256    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n2ltx"
Jul 13 21:29:54.152998 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:54.152913    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:54.153160 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:54.153015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:29:54.153160 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:54.153088    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:29:54.153160 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:54.153107    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f645bcc9-drsxw: secret "image-registry-tls" not found
Jul 13 21:29:54.153160 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:54.153128    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 21:29:54.153160 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:54.153159    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls podName:860f54f5-378e-478c-a414-dc546c02c05b nodeName:}" failed. No retries permitted until 2026-07-13 21:29:55.153144358 +0000 UTC m=+37.075193253 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls") pod "image-registry-79f645bcc9-drsxw" (UID: "860f54f5-378e-478c-a414-dc546c02c05b") : secret "image-registry-tls" not found
Jul 13 21:29:54.153376 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:54.153181    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert podName:8cc99873-ff7d-4ebe-80d8-3147d30440ca nodeName:}" failed. No retries permitted until 2026-07-13 21:29:55.153165042 +0000 UTC m=+37.075213966 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6dsxq" (UID: "8cc99873-ff7d-4ebe-80d8-3147d30440ca") : secret "networking-console-plugin-cert" not found
Jul 13 21:29:54.254147 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:54.254109    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:54.254325 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:54.254261    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:29:54.254369 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:54.254350    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls podName:4418e5fc-2e14-430d-aee2-37c5eeae27bb nodeName:}" failed. No retries permitted until 2026-07-13 21:29:55.254330822 +0000 UTC m=+37.176379718 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls") pod "dns-default-xcd5s" (UID: "4418e5fc-2e14-430d-aee2-37c5eeae27bb") : secret "dns-default-metrics-tls" not found
Jul 13 21:29:54.743553 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:54.743525    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n2ltx"]
Jul 13 21:29:54.747319 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:29:54.747291    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc4ac07c_35b4_46be_8cda_9c35d1fc97bd.slice/crio-5d894be019fe96df4134fc943629fb1114142d8c6fe036a68c5cee3149adda22 WatchSource:0}: Error finding container 5d894be019fe96df4134fc943629fb1114142d8c6fe036a68c5cee3149adda22: Status 404 returned error can't find the container with id 5d894be019fe96df4134fc943629fb1114142d8c6fe036a68c5cee3149adda22
Jul 13 21:29:55.161998 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:55.161914    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:55.161998 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:55.161989    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:29:55.162177 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:55.162061    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:29:55.162177 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:55.162077    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 21:29:55.162177 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:55.162127    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert podName:8cc99873-ff7d-4ebe-80d8-3147d30440ca nodeName:}" failed. No retries permitted until 2026-07-13 21:29:57.162114478 +0000 UTC m=+39.084163372 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6dsxq" (UID: "8cc99873-ff7d-4ebe-80d8-3147d30440ca") : secret "networking-console-plugin-cert" not found
Jul 13 21:29:55.162177 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:55.162080    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f645bcc9-drsxw: secret "image-registry-tls" not found
Jul 13 21:29:55.162311 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:55.162196    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls podName:860f54f5-378e-478c-a414-dc546c02c05b nodeName:}" failed. No retries permitted until 2026-07-13 21:29:57.162179347 +0000 UTC m=+39.084228256 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls") pod "image-registry-79f645bcc9-drsxw" (UID: "860f54f5-378e-478c-a414-dc546c02c05b") : secret "image-registry-tls" not found
Jul 13 21:29:55.263324 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:55.263283    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:55.263494 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:55.263402    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:29:55.263494 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:55.263452    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls podName:4418e5fc-2e14-430d-aee2-37c5eeae27bb nodeName:}" failed. No retries permitted until 2026-07-13 21:29:57.263439664 +0000 UTC m=+39.185488558 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls") pod "dns-default-xcd5s" (UID: "4418e5fc-2e14-430d-aee2-37c5eeae27bb") : secret "dns-default-metrics-tls" not found
Jul 13 21:29:55.701800 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:55.701759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n2ltx" event={"ID":"fc4ac07c-35b4-46be-8cda-9c35d1fc97bd","Type":"ContainerStarted","Data":"5d894be019fe96df4134fc943629fb1114142d8c6fe036a68c5cee3149adda22"}
Jul 13 21:29:55.704928 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:55.704894    2564 generic.go:356] "Generic (PLEG): container finished" podID="e70ee258-310d-4f06-9908-afc7a27684da" containerID="1f34f6f5080eae0b19028fae63c47567839194e82a0f9f5a7b4518ad54ef3c66" exitCode=0
Jul 13 21:29:55.705067 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:55.704947    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlllp" event={"ID":"e70ee258-310d-4f06-9908-afc7a27684da","Type":"ContainerDied","Data":"1f34f6f5080eae0b19028fae63c47567839194e82a0f9f5a7b4518ad54ef3c66"}
Jul 13 21:29:56.711549 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:56.711296    2564 generic.go:356] "Generic (PLEG): container finished" podID="e70ee258-310d-4f06-9908-afc7a27684da" containerID="c83187470f79f76ed80e8fbbb14dc56668358ba1381ce9b6cb9e9772bc620300" exitCode=0
Jul 13 21:29:56.712023 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:56.711384    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlllp" event={"ID":"e70ee258-310d-4f06-9908-afc7a27684da","Type":"ContainerDied","Data":"c83187470f79f76ed80e8fbbb14dc56668358ba1381ce9b6cb9e9772bc620300"}
Jul 13 21:29:57.182527 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:57.182500    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:29:57.182680 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:57.182560    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:29:57.182728 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:57.182675    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 21:29:57.182728 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:57.182676    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:29:57.182794 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:57.182729    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f645bcc9-drsxw: secret "image-registry-tls" not found
Jul 13 21:29:57.182794 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:57.182738    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert podName:8cc99873-ff7d-4ebe-80d8-3147d30440ca nodeName:}" failed. No retries permitted until 2026-07-13 21:30:01.182723935 +0000 UTC m=+43.104772859 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6dsxq" (UID: "8cc99873-ff7d-4ebe-80d8-3147d30440ca") : secret "networking-console-plugin-cert" not found
Jul 13 21:29:57.182794 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:57.182778    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls podName:860f54f5-378e-478c-a414-dc546c02c05b nodeName:}" failed. No retries permitted until 2026-07-13 21:30:01.182763076 +0000 UTC m=+43.104811992 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls") pod "image-registry-79f645bcc9-drsxw" (UID: "860f54f5-378e-478c-a414-dc546c02c05b") : secret "image-registry-tls" not found
Jul 13 21:29:57.283527 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:57.283484    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:29:57.283686 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:57.283631    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:29:57.283739 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:29:57.283715    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls podName:4418e5fc-2e14-430d-aee2-37c5eeae27bb nodeName:}" failed. No retries permitted until 2026-07-13 21:30:01.283696578 +0000 UTC m=+43.205745490 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls") pod "dns-default-xcd5s" (UID: "4418e5fc-2e14-430d-aee2-37c5eeae27bb") : secret "dns-default-metrics-tls" not found
Jul 13 21:29:57.715083 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:57.714990    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n2ltx" event={"ID":"fc4ac07c-35b4-46be-8cda-9c35d1fc97bd","Type":"ContainerStarted","Data":"5be7abd108b775a142d52dc4fcf0b0eb0c4fdc3b0938c103a59c3dfe37e64e9e"}
Jul 13 21:29:57.718081 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:57.718054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlllp" event={"ID":"e70ee258-310d-4f06-9908-afc7a27684da","Type":"ContainerStarted","Data":"acaf9858169b728431f6bf5126825765c92ac228344230b647efadba116cb8dc"}
Jul 13 21:29:57.734397 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:57.734346    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n2ltx" podStartSLOduration=2.169098697 podStartE2EDuration="4.734330861s" totalImagesPullingTime="2.565232164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:57.733593932 +0000 UTC m=+39.655642848" watchObservedRunningTime="2026-07-13 21:29:57.734330861 +0000 UTC m=+39.656379777"
Jul 13 21:29:57.755804 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:29:57.755758    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qlllp" podStartSLOduration=9.46396073 podStartE2EDuration="39.755743751s" totalImagesPullingTime="30.291783021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:29:57.755035019 +0000 UTC m=+39.677083934" watchObservedRunningTime="2026-07-13 21:29:57.755743751 +0000 UTC m=+39.677792668"
Jul 13 21:30:01.213395 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:01.213349    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:30:01.213965 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:01.213451    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:30:01.213965 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:01.213508    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:30:01.213965 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:01.213531    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f645bcc9-drsxw: secret "image-registry-tls" not found
Jul 13 21:30:01.213965 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:01.213583    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 21:30:01.213965 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:01.213589    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls podName:860f54f5-378e-478c-a414-dc546c02c05b nodeName:}" failed. No retries permitted until 2026-07-13 21:30:09.21357337 +0000 UTC m=+51.135622265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls") pod "image-registry-79f645bcc9-drsxw" (UID: "860f54f5-378e-478c-a414-dc546c02c05b") : secret "image-registry-tls" not found
Jul 13 21:30:01.213965 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:01.213669    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert podName:8cc99873-ff7d-4ebe-80d8-3147d30440ca nodeName:}" failed. No retries permitted until 2026-07-13 21:30:09.213631621 +0000 UTC m=+51.135680522 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6dsxq" (UID: "8cc99873-ff7d-4ebe-80d8-3147d30440ca") : secret "networking-console-plugin-cert" not found
Jul 13 21:30:01.314004 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:01.313969    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:30:01.314193 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:01.314103    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:30:01.314193 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:01.314169    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls podName:4418e5fc-2e14-430d-aee2-37c5eeae27bb nodeName:}" failed. No retries permitted until 2026-07-13 21:30:09.314151846 +0000 UTC m=+51.236200740 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls") pod "dns-default-xcd5s" (UID: "4418e5fc-2e14-430d-aee2-37c5eeae27bb") : secret "dns-default-metrics-tls" not found
Jul 13 21:30:09.272503 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:09.272458    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:30:09.272954 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:09.272543    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:30:09.272954 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:09.272608    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:30:09.272954 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:09.272627    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 21:30:09.272954 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:09.272691    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert podName:8cc99873-ff7d-4ebe-80d8-3147d30440ca nodeName:}" failed. No retries permitted until 2026-07-13 21:30:25.272677833 +0000 UTC m=+67.194726727 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6dsxq" (UID: "8cc99873-ff7d-4ebe-80d8-3147d30440ca") : secret "networking-console-plugin-cert" not found
Jul 13 21:30:09.272954 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:09.272628    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f645bcc9-drsxw: secret "image-registry-tls" not found
Jul 13 21:30:09.272954 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:09.272746    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls podName:860f54f5-378e-478c-a414-dc546c02c05b nodeName:}" failed. No retries permitted until 2026-07-13 21:30:25.272728917 +0000 UTC m=+67.194777812 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls") pod "image-registry-79f645bcc9-drsxw" (UID: "860f54f5-378e-478c-a414-dc546c02c05b") : secret "image-registry-tls" not found
Jul 13 21:30:09.373319 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:09.373282    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:30:09.373480 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:09.373424    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:30:09.373519 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:09.373496    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls podName:4418e5fc-2e14-430d-aee2-37c5eeae27bb nodeName:}" failed. No retries permitted until 2026-07-13 21:30:25.373480972 +0000 UTC m=+67.295529866 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls") pod "dns-default-xcd5s" (UID: "4418e5fc-2e14-430d-aee2-37c5eeae27bb") : secret "dns-default-metrics-tls" not found
Jul 13 21:30:19.689044 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:19.689011    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xv5wb"
Jul 13 21:30:24.181249 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.181208    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:30:24.183799 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.183778    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 21:30:24.195092 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.195062    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cb23a38-df98-4abc-9ca3-43fe622a34c7-original-pull-secret\") pod \"global-pull-secret-syncer-nkmbp\" (UID: \"0cb23a38-df98-4abc-9ca3-43fe622a34c7\") " pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:30:24.282110 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.282071    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:30:24.284823 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.284806    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 21:30:24.292728 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:24.292707    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 13 21:30:24.292786 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:24.292776    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs podName:4b89f48b-c81d-40dc-a85d-ea77e98bc1f5 nodeName:}" failed. No retries permitted until 2026-07-13 21:31:28.292760451 +0000 UTC m=+130.214809346 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs") pod "network-metrics-daemon-fjzfw" (UID: "4b89f48b-c81d-40dc-a85d-ea77e98bc1f5") : secret "metrics-daemon-secret" not found
Jul 13 21:30:24.382509 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.382467    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p6xg\" (UniqueName: \"kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg\") pod \"network-check-target-td5gj\" (UID: \"b25ee3f1-7546-49f6-bb7c-6f97b33574f8\") " pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:30:24.385006 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.384987    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 21:30:24.392074 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.392052    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkmbp"
Jul 13 21:30:24.395029 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.395011    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 21:30:24.405975 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.405947    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p6xg\" (UniqueName: \"kubernetes.io/projected/b25ee3f1-7546-49f6-bb7c-6f97b33574f8-kube-api-access-4p6xg\") pod \"network-check-target-td5gj\" (UID: \"b25ee3f1-7546-49f6-bb7c-6f97b33574f8\") " pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:30:24.515363 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.515329    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nkmbp"]
Jul 13 21:30:24.518817 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:30:24.518786    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb23a38_df98_4abc_9ca3_43fe622a34c7.slice/crio-9991c275f7b7969ae9b71fd172b9ca42cfbb5746ca303e9ce9d7a3de19f390f0 WatchSource:0}: Error finding container 9991c275f7b7969ae9b71fd172b9ca42cfbb5746ca303e9ce9d7a3de19f390f0: Status 404 returned error can't find the container with id 9991c275f7b7969ae9b71fd172b9ca42cfbb5746ca303e9ce9d7a3de19f390f0
Jul 13 21:30:24.702466 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.702376    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qsrtw\""
Jul 13 21:30:24.710510 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.710485    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:30:24.783216 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.783192    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nkmbp" event={"ID":"0cb23a38-df98-4abc-9ca3-43fe622a34c7","Type":"ContainerStarted","Data":"9991c275f7b7969ae9b71fd172b9ca42cfbb5746ca303e9ce9d7a3de19f390f0"}
Jul 13 21:30:24.857347 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:24.857313    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-td5gj"]
Jul 13 21:30:24.861072 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:30:24.861048    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25ee3f1_7546_49f6_bb7c_6f97b33574f8.slice/crio-93b9c5e17076fa9bbf10517d4d34507d740c9a9ec70a7fa8937d4f85d1f1ca4a WatchSource:0}: Error finding container 93b9c5e17076fa9bbf10517d4d34507d740c9a9ec70a7fa8937d4f85d1f1ca4a: Status 404 returned error can't find the container with id 93b9c5e17076fa9bbf10517d4d34507d740c9a9ec70a7fa8937d4f85d1f1ca4a
Jul 13 21:30:25.290328 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:25.290267    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:30:25.290817 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:25.290365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:30:25.290817 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:25.290434    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 21:30:25.290817 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:25.290517    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert podName:8cc99873-ff7d-4ebe-80d8-3147d30440ca nodeName:}" failed. No retries permitted until 2026-07-13 21:30:57.29049598 +0000 UTC m=+99.212544878 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6dsxq" (UID: "8cc99873-ff7d-4ebe-80d8-3147d30440ca") : secret "networking-console-plugin-cert" not found
Jul 13 21:30:25.290817 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:25.290525    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:30:25.290817 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:25.290544    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f645bcc9-drsxw: secret "image-registry-tls" not found
Jul 13 21:30:25.290817 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:25.290610    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls podName:860f54f5-378e-478c-a414-dc546c02c05b nodeName:}" failed. No retries permitted until 2026-07-13 21:30:57.290592485 +0000 UTC m=+99.212641390 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls") pod "image-registry-79f645bcc9-drsxw" (UID: "860f54f5-378e-478c-a414-dc546c02c05b") : secret "image-registry-tls" not found
Jul 13 21:30:25.391779 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:25.391741    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:30:25.391982 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:25.391918    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:30:25.392029 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:25.392009    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls podName:4418e5fc-2e14-430d-aee2-37c5eeae27bb nodeName:}" failed. No retries permitted until 2026-07-13 21:30:57.391988059 +0000 UTC m=+99.314036969 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls") pod "dns-default-xcd5s" (UID: "4418e5fc-2e14-430d-aee2-37c5eeae27bb") : secret "dns-default-metrics-tls" not found
Jul 13 21:30:25.787612 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:25.787577    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-td5gj" event={"ID":"b25ee3f1-7546-49f6-bb7c-6f97b33574f8","Type":"ContainerStarted","Data":"93b9c5e17076fa9bbf10517d4d34507d740c9a9ec70a7fa8937d4f85d1f1ca4a"}
Jul 13 21:30:29.801401 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:29.801358    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nkmbp" event={"ID":"0cb23a38-df98-4abc-9ca3-43fe622a34c7","Type":"ContainerStarted","Data":"eb0fa0c482835911d59c6ec37aa3418906465c9ce421fe4afac4ac399395cb3f"}
Jul 13 21:30:29.802768 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:29.802744    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-td5gj" event={"ID":"b25ee3f1-7546-49f6-bb7c-6f97b33574f8","Type":"ContainerStarted","Data":"291ea9c82ff1ab4cab170120e9afcd8ea142f60a1bd4db575611b2943eabcd60"}
Jul 13 21:30:29.802900 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:29.802883    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:30:29.816230 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:29.816190    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nkmbp" podStartSLOduration=67.015808753 podStartE2EDuration="1m11.816174956s" totalImagesPullingTime="4.800366203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:29.815153564 +0000 UTC m=+71.737202483" watchObservedRunningTime="2026-07-13 21:30:29.816174956 +0000 UTC m=+71.738223873"
Jul 13 21:30:29.828734 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:29.828691    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-td5gj" podStartSLOduration=67.375332255 podStartE2EDuration="1m11.828677031s" totalImagesPullingTime="4.453344776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:29.828516367 +0000 UTC m=+71.750565285" watchObservedRunningTime="2026-07-13 21:30:29.828677031 +0000 UTC m=+71.750725950"
Jul 13 21:30:57.319191 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:57.319042    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:30:57.319191 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:57.319099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:30:57.319839 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:57.319199    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:30:57.319839 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:57.319205    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 21:30:57.319839 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:57.319211    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f645bcc9-drsxw: secret "image-registry-tls" not found
Jul 13 21:30:57.319839 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:57.319304    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert podName:8cc99873-ff7d-4ebe-80d8-3147d30440ca nodeName:}" failed. No retries permitted until 2026-07-13 21:32:01.319279506 +0000 UTC m=+163.241328407 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6dsxq" (UID: "8cc99873-ff7d-4ebe-80d8-3147d30440ca") : secret "networking-console-plugin-cert" not found
Jul 13 21:30:57.319839 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:57.319345    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls podName:860f54f5-378e-478c-a414-dc546c02c05b nodeName:}" failed. No retries permitted until 2026-07-13 21:32:01.319333002 +0000 UTC m=+163.241381898 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls") pod "image-registry-79f645bcc9-drsxw" (UID: "860f54f5-378e-478c-a414-dc546c02c05b") : secret "image-registry-tls" not found
Jul 13 21:30:57.419827 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:30:57.419780    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:30:57.419998 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:57.419922    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:30:57.419998 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:30:57.419982    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls podName:4418e5fc-2e14-430d-aee2-37c5eeae27bb nodeName:}" failed. No retries permitted until 2026-07-13 21:32:01.419966246 +0000 UTC m=+163.342015141 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls") pod "dns-default-xcd5s" (UID: "4418e5fc-2e14-430d-aee2-37c5eeae27bb") : secret "dns-default-metrics-tls" not found
Jul 13 21:31:00.808140 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:31:00.808110    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-td5gj"
Jul 13 21:31:28.345856 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:31:28.345795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:31:28.346464 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:31:28.345939    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 13 21:31:28.346464 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:31:28.346003    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs podName:4b89f48b-c81d-40dc-a85d-ea77e98bc1f5 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:30.345987761 +0000 UTC m=+252.268036656 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs") pod "network-metrics-daemon-fjzfw" (UID: "4b89f48b-c81d-40dc-a85d-ea77e98bc1f5") : secret "metrics-daemon-secret" not found
Jul 13 21:31:56.472133 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:31:56.472091    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw" podUID="860f54f5-378e-478c-a414-dc546c02c05b"
Jul 13 21:31:56.484276 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:31:56.484240    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq" podUID="8cc99873-ff7d-4ebe-80d8-3147d30440ca"
Jul 13 21:31:56.506997 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:31:56.506966    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:31:56.513319 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:31:56.513289    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-xcd5s" podUID="4418e5fc-2e14-430d-aee2-37c5eeae27bb"
Jul 13 21:31:57.009331 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:31:57.009299    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:31:57.009520 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:31:57.009300    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:31:57.009520 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:31:57.009306    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:32:01.374622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:32:01.374565    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:32:01.374622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:32:01.374638    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:32:01.375131 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:32:01.374753    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 21:32:01.375131 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:32:01.374770    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:32:01.375131 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:32:01.374782    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f645bcc9-drsxw: secret "image-registry-tls" not found
Jul 13 21:32:01.375131 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:32:01.374832    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls podName:860f54f5-378e-478c-a414-dc546c02c05b nodeName:}" failed. No retries permitted until 2026-07-13 21:34:03.374816735 +0000 UTC m=+285.296865631 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls") pod "image-registry-79f645bcc9-drsxw" (UID: "860f54f5-378e-478c-a414-dc546c02c05b") : secret "image-registry-tls" not found
Jul 13 21:32:01.375131 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:32:01.374844    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert podName:8cc99873-ff7d-4ebe-80d8-3147d30440ca nodeName:}" failed. No retries permitted until 2026-07-13 21:34:03.37483886 +0000 UTC m=+285.296887754 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-6dsxq" (UID: "8cc99873-ff7d-4ebe-80d8-3147d30440ca") : secret "networking-console-plugin-cert" not found
Jul 13 21:32:01.475860 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:32:01.475824    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:32:01.475991 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:32:01.475939    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:32:01.476028 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:32:01.475994    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls podName:4418e5fc-2e14-430d-aee2-37c5eeae27bb nodeName:}" failed. No retries permitted until 2026-07-13 21:34:03.475978349 +0000 UTC m=+285.398027265 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls") pod "dns-default-xcd5s" (UID: "4418e5fc-2e14-430d-aee2-37c5eeae27bb") : secret "dns-default-metrics-tls" not found
Jul 13 21:32:07.467281 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:32:07.467230    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:33:30.417513 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:30.417456    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:33:30.418049 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:30.417605    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 13 21:33:30.418049 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:30.417687    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs podName:4b89f48b-c81d-40dc-a85d-ea77e98bc1f5 nodeName:}" failed. No retries permitted until 2026-07-13 21:35:32.417670645 +0000 UTC m=+374.339719540 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs") pod "network-metrics-daemon-fjzfw" (UID: "4b89f48b-c81d-40dc-a85d-ea77e98bc1f5") : secret "metrics-daemon-secret" not found
Jul 13 21:33:38.435822 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.435785    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-bskbc"]
Jul 13 21:33:38.438717 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.438693    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bskbc"
Jul 13 21:33:38.439167 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.439092    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"]
Jul 13 21:33:38.441448 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.441430    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bjw2b\""
Jul 13 21:33:38.441677 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.441641    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"]
Jul 13 21:33:38.441736 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.441726    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:38.444538 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.444514    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"
Jul 13 21:33:38.444747 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.444723    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 13 21:33:38.445851 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.445508    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-g4f28\""
Jul 13 21:33:38.446287 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.446258    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 13 21:33:38.446425 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.446403    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 13 21:33:38.448244 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.447610    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nrtnh\""
Jul 13 21:33:38.448244 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.447702    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 13 21:33:38.448244 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.448053    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:33:38.448244 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.448053    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 13 21:33:38.449032 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.448626    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 13 21:33:38.452560 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.452533    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-bskbc"]
Jul 13 21:33:38.453623 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.453602    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"]
Jul 13 21:33:38.457276 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.457247    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"]
Jul 13 21:33:38.476410 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.476308    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8rc2\" (UniqueName: \"kubernetes.io/projected/5c267f5a-72ce-4e51-89ad-676469abda4f-kube-api-access-p8rc2\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:38.476410 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.476353    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5c267f5a-72ce-4e51-89ad-676469abda4f-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:38.476410 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.476382    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-4znpw\" (UID: \"d414b68c-474f-44f9-be07-e9f2ac80f2ae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"
Jul 13 21:33:38.476799 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.476495    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w7lp\" (UniqueName: \"kubernetes.io/projected/d414b68c-474f-44f9-be07-e9f2ac80f2ae-kube-api-access-2w7lp\") pod \"cluster-samples-operator-65db858894-4znpw\" (UID: \"d414b68c-474f-44f9-be07-e9f2ac80f2ae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"
Jul 13 21:33:38.476799 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.476549    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:38.476799 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.476668    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7x7x\" (UniqueName: \"kubernetes.io/projected/b1639bfd-b806-4d26-966b-b2361dcd6121-kube-api-access-s7x7x\") pod \"network-check-source-869dcfcdcd-bskbc\" (UID: \"b1639bfd-b806-4d26-966b-b2361dcd6121\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bskbc"
Jul 13 21:33:38.530515 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.530482    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-jcndp"]
Jul 13 21:33:38.533445 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.533427    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-jcndp"
Jul 13 21:33:38.536363 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.536187    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:33:38.536523 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.536477    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"]
Jul 13 21:33:38.536832 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.536807    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-s2dft\""
Jul 13 21:33:38.536942 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.536862    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 13 21:33:38.539341 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.539323    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"]
Jul 13 21:33:38.539470 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.539451    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"
Jul 13 21:33:38.541528 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.541507    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 13 21:33:38.541731 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.541583    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 13 21:33:38.541731 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.541698    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 13 21:33:38.541905 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.541888    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-h5mc6\""
Jul 13 21:33:38.541992 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.541914    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:33:38.542951 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.542927    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-jcndp"]
Jul 13 21:33:38.543073 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.543035    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"
Jul 13 21:33:38.545001 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.544977    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-6jds9\""
Jul 13 21:33:38.545160 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.545142    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 13 21:33:38.545258 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.545108    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 13 21:33:38.545914 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.545898    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 13 21:33:38.546382 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.546356    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:33:38.550456 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.550425    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"]
Jul 13 21:33:38.551534 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.551512    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"]
Jul 13 21:33:38.577092 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577062    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5c267f5a-72ce-4e51-89ad-676469abda4f-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:38.577092 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577099    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6ca58477-dcef-4741-9227-4842c003a667-config\") pod \"service-ca-operator-7695fc6c8c-jwdmx\" (UID: \"6ca58477-dcef-4741-9227-4842c003a667\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"
Jul 13 21:33:38.577324 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577117    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j477\" (UniqueName: \"kubernetes.io/projected/7c0ac9d6-036a-40c0-b399-3270e38f959c-kube-api-access-9j477\") pod \"volume-data-source-validator-78495597f6-jcndp\" (UID: \"7c0ac9d6-036a-40c0-b399-3270e38f959c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-jcndp"
Jul 13 21:33:38.577324 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577137    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w7lp\" (UniqueName: \"kubernetes.io/projected/d414b68c-474f-44f9-be07-e9f2ac80f2ae-kube-api-access-2w7lp\") pod \"cluster-samples-operator-65db858894-4znpw\" (UID: \"d414b68c-474f-44f9-be07-e9f2ac80f2ae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"
Jul 13 21:33:38.577324 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/675ce41d-5f47-4029-aaf7-504ad5f2fc06-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-v9sjz\" (UID: \"675ce41d-5f47-4029-aaf7-504ad5f2fc06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"
Jul 13 21:33:38.577324 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577191    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtz6r\" (UniqueName: \"kubernetes.io/projected/675ce41d-5f47-4029-aaf7-504ad5f2fc06-kube-api-access-jtz6r\") pod \"kube-storage-version-migrator-operator-799cb7f958-v9sjz\" (UID: \"675ce41d-5f47-4029-aaf7-504ad5f2fc06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"
Jul 13 21:33:38.577324 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577215    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ca58477-dcef-4741-9227-4842c003a667-serving-cert\") pod \"service-ca-operator-7695fc6c8c-jwdmx\" (UID: \"6ca58477-dcef-4741-9227-4842c003a667\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"
Jul 13 21:33:38.577324 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-4znpw\" (UID: \"d414b68c-474f-44f9-be07-e9f2ac80f2ae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"
Jul 13 21:33:38.577536 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577331    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npvcm\" (UniqueName: \"kubernetes.io/projected/6ca58477-dcef-4741-9227-4842c003a667-kube-api-access-npvcm\") pod \"service-ca-operator-7695fc6c8c-jwdmx\" (UID: \"6ca58477-dcef-4741-9227-4842c003a667\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"
Jul 13 21:33:38.577536 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577355    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/675ce41d-5f47-4029-aaf7-504ad5f2fc06-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-v9sjz\" (UID: \"675ce41d-5f47-4029-aaf7-504ad5f2fc06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"
Jul 13 21:33:38.577536 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577409    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:38.577536 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:38.577458    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 21:33:38.577536 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577489    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7x7x\" (UniqueName: \"kubernetes.io/projected/b1639bfd-b806-4d26-966b-b2361dcd6121-kube-api-access-s7x7x\") pod \"network-check-source-869dcfcdcd-bskbc\" (UID: \"b1639bfd-b806-4d26-966b-b2361dcd6121\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bskbc"
Jul 13 21:33:38.577536 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:38.577509    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 21:33:38.577536 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:38.577514    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls podName:d414b68c-474f-44f9-be07-e9f2ac80f2ae nodeName:}" failed. No retries permitted until 2026-07-13 21:33:39.0774955 +0000 UTC m=+260.999544406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls") pod "cluster-samples-operator-65db858894-4znpw" (UID: "d414b68c-474f-44f9-be07-e9f2ac80f2ae") : secret "samples-operator-tls" not found
Jul 13 21:33:38.577876 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577545    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8rc2\" (UniqueName: \"kubernetes.io/projected/5c267f5a-72ce-4e51-89ad-676469abda4f-kube-api-access-p8rc2\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:38.577876 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:38.577577    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls podName:5c267f5a-72ce-4e51-89ad-676469abda4f nodeName:}" failed. No retries permitted until 2026-07-13 21:33:39.077564441 +0000 UTC m=+260.999613335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-h685t" (UID: "5c267f5a-72ce-4e51-89ad-676469abda4f") : secret "cluster-monitoring-operator-tls" not found
Jul 13 21:33:38.577945 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.577883    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5c267f5a-72ce-4e51-89ad-676469abda4f-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:38.590576 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.590538    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7x7x\" (UniqueName: \"kubernetes.io/projected/b1639bfd-b806-4d26-966b-b2361dcd6121-kube-api-access-s7x7x\") pod \"network-check-source-869dcfcdcd-bskbc\" (UID: \"b1639bfd-b806-4d26-966b-b2361dcd6121\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bskbc"
Jul 13 21:33:38.590773 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.590603    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8rc2\" (UniqueName: \"kubernetes.io/projected/5c267f5a-72ce-4e51-89ad-676469abda4f-kube-api-access-p8rc2\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:38.590773 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.590687    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w7lp\" (UniqueName: \"kubernetes.io/projected/d414b68c-474f-44f9-be07-e9f2ac80f2ae-kube-api-access-2w7lp\") pod \"cluster-samples-operator-65db858894-4znpw\" (UID: \"d414b68c-474f-44f9-be07-e9f2ac80f2ae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"
Jul 13 21:33:38.678551 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.678491    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npvcm\" (UniqueName: \"kubernetes.io/projected/6ca58477-dcef-4741-9227-4842c003a667-kube-api-access-npvcm\") pod \"service-ca-operator-7695fc6c8c-jwdmx\" (UID: \"6ca58477-dcef-4741-9227-4842c003a667\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"
Jul 13 21:33:38.678551 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.678548    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/675ce41d-5f47-4029-aaf7-504ad5f2fc06-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-v9sjz\" (UID: \"675ce41d-5f47-4029-aaf7-504ad5f2fc06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"
Jul 13 21:33:38.678848 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.678618    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6ca58477-dcef-4741-9227-4842c003a667-config\") pod \"service-ca-operator-7695fc6c8c-jwdmx\" (UID: \"6ca58477-dcef-4741-9227-4842c003a667\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"
Jul 13 21:33:38.678848 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.678644    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j477\" (UniqueName: \"kubernetes.io/projected/7c0ac9d6-036a-40c0-b399-3270e38f959c-kube-api-access-9j477\") pod \"volume-data-source-validator-78495597f6-jcndp\" (UID: \"7c0ac9d6-036a-40c0-b399-3270e38f959c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-jcndp"
Jul 13 21:33:38.678848 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.678693    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/675ce41d-5f47-4029-aaf7-504ad5f2fc06-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-v9sjz\" (UID: \"675ce41d-5f47-4029-aaf7-504ad5f2fc06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"
Jul 13 21:33:38.678848 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.678722    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtz6r\" (UniqueName: \"kubernetes.io/projected/675ce41d-5f47-4029-aaf7-504ad5f2fc06-kube-api-access-jtz6r\") pod \"kube-storage-version-migrator-operator-799cb7f958-v9sjz\" (UID: \"675ce41d-5f47-4029-aaf7-504ad5f2fc06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"
Jul 13 21:33:38.678848 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.678744    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ca58477-dcef-4741-9227-4842c003a667-serving-cert\") pod \"service-ca-operator-7695fc6c8c-jwdmx\" (UID: \"6ca58477-dcef-4741-9227-4842c003a667\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"
Jul 13 21:33:38.679306 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.679269    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6ca58477-dcef-4741-9227-4842c003a667-config\") pod \"service-ca-operator-7695fc6c8c-jwdmx\" (UID: \"6ca58477-dcef-4741-9227-4842c003a667\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"
Jul 13 21:33:38.679421 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.679269    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/675ce41d-5f47-4029-aaf7-504ad5f2fc06-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-v9sjz\" (UID: \"675ce41d-5f47-4029-aaf7-504ad5f2fc06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"
Jul 13 21:33:38.680863 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.680842    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/675ce41d-5f47-4029-aaf7-504ad5f2fc06-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-v9sjz\" (UID: \"675ce41d-5f47-4029-aaf7-504ad5f2fc06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"
Jul 13 21:33:38.680922 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.680880    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ca58477-dcef-4741-9227-4842c003a667-serving-cert\") pod \"service-ca-operator-7695fc6c8c-jwdmx\" (UID: \"6ca58477-dcef-4741-9227-4842c003a667\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"
Jul 13 21:33:38.686418 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.686356    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npvcm\" (UniqueName: \"kubernetes.io/projected/6ca58477-dcef-4741-9227-4842c003a667-kube-api-access-npvcm\") pod \"service-ca-operator-7695fc6c8c-jwdmx\" (UID: \"6ca58477-dcef-4741-9227-4842c003a667\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"
Jul 13 21:33:38.686534 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.686496    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j477\" (UniqueName: \"kubernetes.io/projected/7c0ac9d6-036a-40c0-b399-3270e38f959c-kube-api-access-9j477\") pod \"volume-data-source-validator-78495597f6-jcndp\" (UID: \"7c0ac9d6-036a-40c0-b399-3270e38f959c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-jcndp"
Jul 13 21:33:38.686534 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.686496    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtz6r\" (UniqueName: \"kubernetes.io/projected/675ce41d-5f47-4029-aaf7-504ad5f2fc06-kube-api-access-jtz6r\") pod \"kube-storage-version-migrator-operator-799cb7f958-v9sjz\" (UID: \"675ce41d-5f47-4029-aaf7-504ad5f2fc06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"
Jul 13 21:33:38.753664 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.753586    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bskbc"
Jul 13 21:33:38.844111 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.844078    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-jcndp"
Jul 13 21:33:38.854131 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.854104    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"
Jul 13 21:33:38.860197 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.860173    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"
Jul 13 21:33:38.882832 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:38.882801    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-bskbc"]
Jul 13 21:33:38.886244 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:33:38.886218    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1639bfd_b806_4d26_966b_b2361dcd6121.slice/crio-5da9879a22140077d565795cbc31ac4efb723a4b775bd821596bdd7d0cc1001c WatchSource:0}: Error finding container 5da9879a22140077d565795cbc31ac4efb723a4b775bd821596bdd7d0cc1001c: Status 404 returned error can't find the container with id 5da9879a22140077d565795cbc31ac4efb723a4b775bd821596bdd7d0cc1001c
Jul 13 21:33:39.005005 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:39.004920    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-jcndp"]
Jul 13 21:33:39.016209 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:39.016161    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx"]
Jul 13 21:33:39.020410 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:33:39.020372    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ca58477_dcef_4741_9227_4842c003a667.slice/crio-cd751621778977ff227e1e2c23bddcf775b0f8f19821cef3e4c9c71c3078aa28 WatchSource:0}: Error finding container cd751621778977ff227e1e2c23bddcf775b0f8f19821cef3e4c9c71c3078aa28: Status 404 returned error can't find the container with id cd751621778977ff227e1e2c23bddcf775b0f8f19821cef3e4c9c71c3078aa28
Jul 13 21:33:39.034641 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:39.034608    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz"]
Jul 13 21:33:39.037708 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:33:39.037672    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod675ce41d_5f47_4029_aaf7_504ad5f2fc06.slice/crio-f476ff7781f4ce79320b29e10023630fbf00b3f9870cd96082dae7229c1d1b0c WatchSource:0}: Error finding container f476ff7781f4ce79320b29e10023630fbf00b3f9870cd96082dae7229c1d1b0c: Status 404 returned error can't find the container with id f476ff7781f4ce79320b29e10023630fbf00b3f9870cd96082dae7229c1d1b0c
Jul 13 21:33:39.081736 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:39.081699    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-4znpw\" (UID: \"d414b68c-474f-44f9-be07-e9f2ac80f2ae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"
Jul 13 21:33:39.081736 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:39.081741    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:39.081921 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:39.081850    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 21:33:39.081921 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:39.081917    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls podName:5c267f5a-72ce-4e51-89ad-676469abda4f nodeName:}" failed. No retries permitted until 2026-07-13 21:33:40.081902619 +0000 UTC m=+262.003951515 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-h685t" (UID: "5c267f5a-72ce-4e51-89ad-676469abda4f") : secret "cluster-monitoring-operator-tls" not found
Jul 13 21:33:39.082017 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:39.081850    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 21:33:39.082017 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:39.081987    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls podName:d414b68c-474f-44f9-be07-e9f2ac80f2ae nodeName:}" failed. No retries permitted until 2026-07-13 21:33:40.081973479 +0000 UTC m=+262.004022374 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls") pod "cluster-samples-operator-65db858894-4znpw" (UID: "d414b68c-474f-44f9-be07-e9f2ac80f2ae") : secret "samples-operator-tls" not found
Jul 13 21:33:39.249794 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:39.249706    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-jcndp" event={"ID":"7c0ac9d6-036a-40c0-b399-3270e38f959c","Type":"ContainerStarted","Data":"cb5d0383f8f0700ef641122c0b63e455e584b89da1bcf407f58b2169b274b028"}
Jul 13 21:33:39.253429 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:39.253386    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx" event={"ID":"6ca58477-dcef-4741-9227-4842c003a667","Type":"ContainerStarted","Data":"cd751621778977ff227e1e2c23bddcf775b0f8f19821cef3e4c9c71c3078aa28"}
Jul 13 21:33:39.254906 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:39.254877    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bskbc" event={"ID":"b1639bfd-b806-4d26-966b-b2361dcd6121","Type":"ContainerStarted","Data":"631da575d089265b4e3e61694d62e60f925c01421f46369c3c652c92eec8ff97"}
Jul 13 21:33:39.255066 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:39.254927    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bskbc" event={"ID":"b1639bfd-b806-4d26-966b-b2361dcd6121","Type":"ContainerStarted","Data":"5da9879a22140077d565795cbc31ac4efb723a4b775bd821596bdd7d0cc1001c"}
Jul 13 21:33:39.255931 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:39.255902    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz" event={"ID":"675ce41d-5f47-4029-aaf7-504ad5f2fc06","Type":"ContainerStarted","Data":"f476ff7781f4ce79320b29e10023630fbf00b3f9870cd96082dae7229c1d1b0c"}
Jul 13 21:33:39.269930 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:39.269878    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bskbc" podStartSLOduration=1.2698607929999999 podStartE2EDuration="1.269860793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:33:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:33:39.269046475 +0000 UTC m=+261.191095394" watchObservedRunningTime="2026-07-13 21:33:39.269860793 +0000 UTC m=+261.191909709"
Jul 13 21:33:40.092427 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:40.091758    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-4znpw\" (UID: \"d414b68c-474f-44f9-be07-e9f2ac80f2ae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"
Jul 13 21:33:40.092427 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:40.091822    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:40.092427 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:40.092004    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 21:33:40.092427 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:40.092068    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls podName:5c267f5a-72ce-4e51-89ad-676469abda4f nodeName:}" failed. No retries permitted until 2026-07-13 21:33:42.092048526 +0000 UTC m=+264.014097427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-h685t" (UID: "5c267f5a-72ce-4e51-89ad-676469abda4f") : secret "cluster-monitoring-operator-tls" not found
Jul 13 21:33:40.092427 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:40.092205    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 21:33:40.092427 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:40.092252    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls podName:d414b68c-474f-44f9-be07-e9f2ac80f2ae nodeName:}" failed. No retries permitted until 2026-07-13 21:33:42.092238916 +0000 UTC m=+264.014287815 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls") pod "cluster-samples-operator-65db858894-4znpw" (UID: "d414b68c-474f-44f9-be07-e9f2ac80f2ae") : secret "samples-operator-tls" not found
Jul 13 21:33:41.264607 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:41.264570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-jcndp" event={"ID":"7c0ac9d6-036a-40c0-b399-3270e38f959c","Type":"ContainerStarted","Data":"3f40dc1022adea62fe4200c45cb7e686193eac365285ed46762f699205af2b09"}
Jul 13 21:33:41.279804 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:41.279743    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-jcndp" podStartSLOduration=2.06723889 podStartE2EDuration="3.279727839s" totalImagesPullingTime="1.212488949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:33:41.279063006 +0000 UTC m=+263.201111923" watchObservedRunningTime="2026-07-13 21:33:41.279727839 +0000 UTC m=+263.201776755"
Jul 13 21:33:42.111458 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:42.111414    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:42.111665 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:42.111563    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-4znpw\" (UID: \"d414b68c-474f-44f9-be07-e9f2ac80f2ae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"
Jul 13 21:33:42.111665 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:42.111582    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 21:33:42.111763 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:42.111682    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls podName:5c267f5a-72ce-4e51-89ad-676469abda4f nodeName:}" failed. No retries permitted until 2026-07-13 21:33:46.111660779 +0000 UTC m=+268.033709693 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-h685t" (UID: "5c267f5a-72ce-4e51-89ad-676469abda4f") : secret "cluster-monitoring-operator-tls" not found
Jul 13 21:33:42.111763 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:42.111707    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 21:33:42.111763 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:42.111759    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls podName:d414b68c-474f-44f9-be07-e9f2ac80f2ae nodeName:}" failed. No retries permitted until 2026-07-13 21:33:46.111742869 +0000 UTC m=+268.033791770 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls") pod "cluster-samples-operator-65db858894-4znpw" (UID: "d414b68c-474f-44f9-be07-e9f2ac80f2ae") : secret "samples-operator-tls" not found
Jul 13 21:33:42.270194 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:42.270154    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz" event={"ID":"675ce41d-5f47-4029-aaf7-504ad5f2fc06","Type":"ContainerStarted","Data":"332f7f3c842d5ba3e0fbd55f195c6964281015ce55c408c980df5f4348b20e2d"}
Jul 13 21:33:42.271838 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:42.271810    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx" event={"ID":"6ca58477-dcef-4741-9227-4842c003a667","Type":"ContainerStarted","Data":"edb7d52811cc54c6a4c89338637ff727eb6987117dd95a4ccb99ddb8178a84b9"}
Jul 13 21:33:42.287877 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:42.287823    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz" podStartSLOduration=1.576675104 podStartE2EDuration="4.287806587s" totalImagesPullingTime="2.711131483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:33:42.286730313 +0000 UTC m=+264.208779232" watchObservedRunningTime="2026-07-13 21:33:42.287806587 +0000 UTC m=+264.209855537"
Jul 13 21:33:42.302186 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:42.302129    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx" podStartSLOduration=1.571242699 podStartE2EDuration="4.302107956s" totalImagesPullingTime="2.730865257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:33:42.300893114 +0000 UTC m=+264.222942030" watchObservedRunningTime="2026-07-13 21:33:42.302107956 +0000 UTC m=+264.224156878"
Jul 13 21:33:43.364889 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.364853    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-twl8l"]
Jul 13 21:33:43.368147 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.368128    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-twl8l"
Jul 13 21:33:43.370781 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.370755    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-tspt6\""
Jul 13 21:33:43.370949 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.370795    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 13 21:33:43.371568 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.371551    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 13 21:33:43.376659 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.376619    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-twl8l"]
Jul 13 21:33:43.421807 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.421762    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8l5q\" (UniqueName: \"kubernetes.io/projected/f4d535ca-9697-4610-9dba-cf0965688aaf-kube-api-access-r8l5q\") pod \"migrator-7f59b54f6c-twl8l\" (UID: \"f4d535ca-9697-4610-9dba-cf0965688aaf\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-twl8l"
Jul 13 21:33:43.522854 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.522808    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8l5q\" (UniqueName: \"kubernetes.io/projected/f4d535ca-9697-4610-9dba-cf0965688aaf-kube-api-access-r8l5q\") pod \"migrator-7f59b54f6c-twl8l\" (UID: \"f4d535ca-9697-4610-9dba-cf0965688aaf\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-twl8l"
Jul 13 21:33:43.530569 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.530531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8l5q\" (UniqueName: \"kubernetes.io/projected/f4d535ca-9697-4610-9dba-cf0965688aaf-kube-api-access-r8l5q\") pod \"migrator-7f59b54f6c-twl8l\" (UID: \"f4d535ca-9697-4610-9dba-cf0965688aaf\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-twl8l"
Jul 13 21:33:43.678718 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.678612    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-twl8l"
Jul 13 21:33:43.784922 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.784896    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cnzqd"]
Jul 13 21:33:43.788252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.788232    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:43.790856 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.790835    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 13 21:33:43.791024 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.790855    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tf6cc\""
Jul 13 21:33:43.791024 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.790988    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 13 21:33:43.791199 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.791172    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 13 21:33:43.791771 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.791748    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 13 21:33:43.799339 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.799312    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cnzqd"]
Jul 13 21:33:43.817960 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.817927    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-twl8l"]
Jul 13 21:33:43.821053 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:33:43.821018    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d535ca_9697_4610_9dba_cf0965688aaf.slice/crio-34ba613b22860e62ef4c6f266fead1dbfafca7b48766050854ffa0dd8d833360 WatchSource:0}: Error finding container 34ba613b22860e62ef4c6f266fead1dbfafca7b48766050854ffa0dd8d833360: Status 404 returned error can't find the container with id 34ba613b22860e62ef4c6f266fead1dbfafca7b48766050854ffa0dd8d833360
Jul 13 21:33:43.927045 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.927012    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:43.927045 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.927046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpcrz\" (UniqueName: \"kubernetes.io/projected/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-api-access-cpcrz\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:43.927266 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.927070    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:43.927266 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.927167    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70c39abc-b85c-4453-95ad-d9ca15d1b706-crio-socket\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:43.927266 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:43.927213    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70c39abc-b85c-4453-95ad-d9ca15d1b706-data-volume\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:44.028020 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:44.027982    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:44.028191 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:44.028028    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpcrz\" (UniqueName: \"kubernetes.io/projected/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-api-access-cpcrz\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:44.028191 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:44.028061    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:44.028191 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:44.028123    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 21:33:44.028191 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:44.028163    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 21:33:44.028325 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:44.028214    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls podName:70c39abc-b85c-4453-95ad-d9ca15d1b706 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:44.528194539 +0000 UTC m=+266.450243434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cnzqd" (UID: "70c39abc-b85c-4453-95ad-d9ca15d1b706") : secret "insights-runtime-extractor-tls" not found
Jul 13 21:33:44.028325 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:44.028240    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm podName:70c39abc-b85c-4453-95ad-d9ca15d1b706 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:44.528229887 +0000 UTC m=+266.450278807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cnzqd" (UID: "70c39abc-b85c-4453-95ad-d9ca15d1b706") : configmap "kube-rbac-proxy" not found
Jul 13 21:33:44.028325 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:44.028279    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70c39abc-b85c-4453-95ad-d9ca15d1b706-crio-socket\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:44.028325 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:44.028318    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70c39abc-b85c-4453-95ad-d9ca15d1b706-data-volume\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:44.028458 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:44.028406    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/70c39abc-b85c-4453-95ad-d9ca15d1b706-crio-socket\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:44.028620 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:44.028604    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/70c39abc-b85c-4453-95ad-d9ca15d1b706-data-volume\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:44.036039 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:44.036011    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpcrz\" (UniqueName: \"kubernetes.io/projected/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-api-access-cpcrz\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:44.279644 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:44.279560    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-twl8l" event={"ID":"f4d535ca-9697-4610-9dba-cf0965688aaf","Type":"ContainerStarted","Data":"34ba613b22860e62ef4c6f266fead1dbfafca7b48766050854ffa0dd8d833360"}
Jul 13 21:33:44.533120 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:44.532851    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:44.533120 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:44.532911    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:44.533120 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:44.533032    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 21:33:44.533120 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:44.533103    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm podName:70c39abc-b85c-4453-95ad-d9ca15d1b706 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:45.533082491 +0000 UTC m=+267.455131400 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cnzqd" (UID: "70c39abc-b85c-4453-95ad-d9ca15d1b706") : configmap "kube-rbac-proxy" not found
Jul 13 21:33:44.533120 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:44.533112    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 21:33:44.533730 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:44.533167    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls podName:70c39abc-b85c-4453-95ad-d9ca15d1b706 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:45.533149967 +0000 UTC m=+267.455198870 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cnzqd" (UID: "70c39abc-b85c-4453-95ad-d9ca15d1b706") : secret "insights-runtime-extractor-tls" not found
Jul 13 21:33:45.536313 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.536274    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qnft2"]
Jul 13 21:33:45.539336 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.539319    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2"
Jul 13 21:33:45.541682 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.541627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:45.541793 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.541701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:45.541793 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.541743    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 13 21:33:45.541793 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:45.541751    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 21:33:45.541952 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.541743    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9lxqm\""
Jul 13 21:33:45.541952 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:45.541825    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm podName:70c39abc-b85c-4453-95ad-d9ca15d1b706 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:47.541806241 +0000 UTC m=+269.463855143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cnzqd" (UID: "70c39abc-b85c-4453-95ad-d9ca15d1b706") : configmap "kube-rbac-proxy" not found
Jul 13 21:33:45.541952 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.541828    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 13 21:33:45.541952 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:45.541864    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 21:33:45.541952 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:45.541911    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls podName:70c39abc-b85c-4453-95ad-d9ca15d1b706 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:47.541892222 +0000 UTC m=+269.463941117 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cnzqd" (UID: "70c39abc-b85c-4453-95ad-d9ca15d1b706") : secret "insights-runtime-extractor-tls" not found
Jul 13 21:33:45.542405 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.542388    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 13 21:33:45.542447 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.542391    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 13 21:33:45.545797 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.545776    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qnft2"]
Jul 13 21:33:45.642497 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.642455    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8vtr\" (UniqueName: \"kubernetes.io/projected/f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1-kube-api-access-p8vtr\") pod \"service-ca-79f68b9f4f-qnft2\" (UID: \"f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2"
Jul 13 21:33:45.642704 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.642535    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1-signing-cabundle\") pod \"service-ca-79f68b9f4f-qnft2\" (UID: \"f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2"
Jul 13 21:33:45.642704 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.642582    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1-signing-key\") pod \"service-ca-79f68b9f4f-qnft2\" (UID: \"f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2"
Jul 13 21:33:45.747850 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.747809    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8vtr\" (UniqueName: \"kubernetes.io/projected/f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1-kube-api-access-p8vtr\") pod \"service-ca-79f68b9f4f-qnft2\" (UID: \"f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2"
Jul 13 21:33:45.748020 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.747886    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1-signing-cabundle\") pod \"service-ca-79f68b9f4f-qnft2\" (UID: \"f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2"
Jul 13 21:33:45.748020 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.747923    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1-signing-key\") pod \"service-ca-79f68b9f4f-qnft2\" (UID: \"f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2"
Jul 13 21:33:45.748721 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.748687    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1-signing-cabundle\") pod \"service-ca-79f68b9f4f-qnft2\" (UID: \"f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2"
Jul 13 21:33:45.750292 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.750265    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1-signing-key\") pod \"service-ca-79f68b9f4f-qnft2\" (UID: \"f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2"
Jul 13 21:33:45.756336 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.756303    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8vtr\" (UniqueName: \"kubernetes.io/projected/f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1-kube-api-access-p8vtr\") pod \"service-ca-79f68b9f4f-qnft2\" (UID: \"f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2"
Jul 13 21:33:45.849426 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.849329    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2"
Jul 13 21:33:45.975512 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:45.975474    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qnft2"]
Jul 13 21:33:45.978360 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:33:45.978330    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1c6cc29_20d3_4e83_92c2_f9c57fddc0d1.slice/crio-bcf5a6e5fbb5a07b10da983f0512d501936fd69ac3cdb7f7236d23d2d1313cae WatchSource:0}: Error finding container bcf5a6e5fbb5a07b10da983f0512d501936fd69ac3cdb7f7236d23d2d1313cae: Status 404 returned error can't find the container with id bcf5a6e5fbb5a07b10da983f0512d501936fd69ac3cdb7f7236d23d2d1313cae
Jul 13 21:33:46.151263 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:46.151168    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-4znpw\" (UID: \"d414b68c-474f-44f9-be07-e9f2ac80f2ae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"
Jul 13 21:33:46.151263 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:46.151228    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:46.151478 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:46.151344    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 21:33:46.151478 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:46.151383    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 21:33:46.151478 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:46.151428    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls podName:d414b68c-474f-44f9-be07-e9f2ac80f2ae nodeName:}" failed. No retries permitted until 2026-07-13 21:33:54.151405499 +0000 UTC m=+276.073454401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls") pod "cluster-samples-operator-65db858894-4znpw" (UID: "d414b68c-474f-44f9-be07-e9f2ac80f2ae") : secret "samples-operator-tls" not found
Jul 13 21:33:46.151478 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:46.151447    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls podName:5c267f5a-72ce-4e51-89ad-676469abda4f nodeName:}" failed. No retries permitted until 2026-07-13 21:33:54.151437597 +0000 UTC m=+276.073486494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-h685t" (UID: "5c267f5a-72ce-4e51-89ad-676469abda4f") : secret "cluster-monitoring-operator-tls" not found
Jul 13 21:33:46.288870 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:46.288834    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-twl8l" event={"ID":"f4d535ca-9697-4610-9dba-cf0965688aaf","Type":"ContainerStarted","Data":"9d197d95ab867892210a1e9b508a44e3e24dc74cc952f1a4c493d5d815226e82"}
Jul 13 21:33:46.289077 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:46.288898    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-twl8l" event={"ID":"f4d535ca-9697-4610-9dba-cf0965688aaf","Type":"ContainerStarted","Data":"a5e057fe0f0cf6e57a42d8caec7caca3d52fa663cd37f0f0ea88ad430a7746ce"}
Jul 13 21:33:46.290467 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:46.290439    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2" event={"ID":"f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1","Type":"ContainerStarted","Data":"47f4868f7795111b18192a124b1cba5ef55d5ccdc95ad6796dde20c2fcc2a37f"}
Jul 13 21:33:46.290576 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:46.290487    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2" event={"ID":"f1c6cc29-20d3-4e83-92c2-f9c57fddc0d1","Type":"ContainerStarted","Data":"bcf5a6e5fbb5a07b10da983f0512d501936fd69ac3cdb7f7236d23d2d1313cae"}
Jul 13 21:33:46.304907 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:46.304855    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-twl8l" podStartSLOduration=1.884996202 podStartE2EDuration="3.30483726s" totalImagesPullingTime="1.419841058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:33:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:33:46.303745621 +0000 UTC m=+268.225794538" watchObservedRunningTime="2026-07-13 21:33:46.30483726 +0000 UTC m=+268.226886177"
Jul 13 21:33:46.318999 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:46.318940    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-qnft2" podStartSLOduration=1.3189244869999999 podStartE2EDuration="1.318924487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:33:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:33:46.31870279 +0000 UTC m=+268.240751708" watchObservedRunningTime="2026-07-13 21:33:46.318924487 +0000 UTC m=+268.240973404"
Jul 13 21:33:46.388902 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:46.388871    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pt6c7_53433330-3378-4829-be11-5a219e621ef4/dns-node-resolver/0.log"
Jul 13 21:33:47.388117 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:47.388060    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4v4mk_c8726c19-73c9-4166-97a2-88f09f2bb28c/node-ca/0.log"
Jul 13 21:33:47.563804 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:47.563751    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:47.564007 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:47.563815    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:47.564007 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:47.563975    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 21:33:47.564115 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:47.564037    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls podName:70c39abc-b85c-4453-95ad-d9ca15d1b706 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:51.564018244 +0000 UTC m=+273.486067145 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cnzqd" (UID: "70c39abc-b85c-4453-95ad-d9ca15d1b706") : secret "insights-runtime-extractor-tls" not found
Jul 13 21:33:47.564300 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:47.564281    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 21:33:47.564435 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:47.564424    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm podName:70c39abc-b85c-4453-95ad-d9ca15d1b706 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:51.564406969 +0000 UTC m=+273.486455867 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cnzqd" (UID: "70c39abc-b85c-4453-95ad-d9ca15d1b706") : configmap "kube-rbac-proxy" not found
Jul 13 21:33:48.588456 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:48.588422    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n2ltx_fc4ac07c-35b4-46be-8cda-9c35d1fc97bd/serve-healthcheck-canary/0.log"
Jul 13 21:33:48.788388 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:48.788356    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-twl8l_f4d535ca-9697-4610-9dba-cf0965688aaf/migrator/0.log"
Jul 13 21:33:48.989638 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:48.989605    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-twl8l_f4d535ca-9697-4610-9dba-cf0965688aaf/graceful-termination/0.log"
Jul 13 21:33:49.190430 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:49.190353    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-v9sjz_675ce41d-5f47-4029-aaf7-504ad5f2fc06/kube-storage-version-migrator-operator/0.log"
Jul 13 21:33:51.601248 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:51.601209    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:51.601248 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:51.601257    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:51.601682 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:51.601335    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 21:33:51.601682 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:51.601381    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 21:33:51.601682 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:51.601398    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm podName:70c39abc-b85c-4453-95ad-d9ca15d1b706 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:59.601383236 +0000 UTC m=+281.523432132 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cnzqd" (UID: "70c39abc-b85c-4453-95ad-d9ca15d1b706") : configmap "kube-rbac-proxy" not found
Jul 13 21:33:51.601682 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:51.601426    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls podName:70c39abc-b85c-4453-95ad-d9ca15d1b706 nodeName:}" failed. No retries permitted until 2026-07-13 21:33:59.6014134 +0000 UTC m=+281.523462296 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cnzqd" (UID: "70c39abc-b85c-4453-95ad-d9ca15d1b706") : secret "insights-runtime-extractor-tls" not found
Jul 13 21:33:54.224999 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:54.224958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-4znpw\" (UID: \"d414b68c-474f-44f9-be07-e9f2ac80f2ae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"
Jul 13 21:33:54.225431 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:54.225026    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:33:54.225431 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:54.225178    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 21:33:54.225431 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:33:54.225246    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls podName:5c267f5a-72ce-4e51-89ad-676469abda4f nodeName:}" failed. No retries permitted until 2026-07-13 21:34:10.225228567 +0000 UTC m=+292.147277467 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-h685t" (UID: "5c267f5a-72ce-4e51-89ad-676469abda4f") : secret "cluster-monitoring-operator-tls" not found
Jul 13 21:33:54.227394 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:54.227371    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d414b68c-474f-44f9-be07-e9f2ac80f2ae-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-4znpw\" (UID: \"d414b68c-474f-44f9-be07-e9f2ac80f2ae\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"
Jul 13 21:33:54.368220 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:54.368175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"
Jul 13 21:33:54.497252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:54.497168    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw"]
Jul 13 21:33:55.319742 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:55.319688    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw" event={"ID":"d414b68c-474f-44f9-be07-e9f2ac80f2ae","Type":"ContainerStarted","Data":"1a9ae11880d641ca65e8c785eb9d867c7ada5b171bbf3cf14c2cf592fece743b"}
Jul 13 21:33:57.328309 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:57.328269    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw" event={"ID":"d414b68c-474f-44f9-be07-e9f2ac80f2ae","Type":"ContainerStarted","Data":"13ed9ca3ded7b64b79b229e40381929b756ac739dbf59b503b921664a387fcc8"}
Jul 13 21:33:57.328820 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:57.328325    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw" event={"ID":"d414b68c-474f-44f9-be07-e9f2ac80f2ae","Type":"ContainerStarted","Data":"f3ba868417b3b5ad6219474a2fbd47cf1610ecc15780daaf28ed7dc455e56ac4"}
Jul 13 21:33:57.345229 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:57.345179    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-4znpw" podStartSLOduration=17.511215508 podStartE2EDuration="19.345165524s" totalImagesPullingTime="1.833950016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:33:57.343410662 +0000 UTC m=+279.265459578" watchObservedRunningTime="2026-07-13 21:33:57.345165524 +0000 UTC m=+279.267214460"
Jul 13 21:33:59.675026 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:59.674983    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:59.675026 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:59.675026    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:59.675709 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:59.675691    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/70c39abc-b85c-4453-95ad-d9ca15d1b706-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:59.677269 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:59.677252    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/70c39abc-b85c-4453-95ad-d9ca15d1b706-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cnzqd\" (UID: \"70c39abc-b85c-4453-95ad-d9ca15d1b706\") " pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:59.698445 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:59.698410    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cnzqd"
Jul 13 21:33:59.846302 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:33:59.846102    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cnzqd"]
Jul 13 21:33:59.848926 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:33:59.848891    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c39abc_b85c_4453_95ad_d9ca15d1b706.slice/crio-3b91df8f59344b30dda779157a09f8400ff205a775a540de5f515170c963add2 WatchSource:0}: Error finding container 3b91df8f59344b30dda779157a09f8400ff205a775a540de5f515170c963add2: Status 404 returned error can't find the container with id 3b91df8f59344b30dda779157a09f8400ff205a775a540de5f515170c963add2
Jul 13 21:34:00.010477 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:34:00.010430    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq" podUID="8cc99873-ff7d-4ebe-80d8-3147d30440ca"
Jul 13 21:34:00.010477 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:34:00.010445    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw" podUID="860f54f5-378e-478c-a414-dc546c02c05b"
Jul 13 21:34:00.010477 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:34:00.010449    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-xcd5s" podUID="4418e5fc-2e14-430d-aee2-37c5eeae27bb"
Jul 13 21:34:00.339621 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:00.339522    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cnzqd" event={"ID":"70c39abc-b85c-4453-95ad-d9ca15d1b706","Type":"ContainerStarted","Data":"3b91df8f59344b30dda779157a09f8400ff205a775a540de5f515170c963add2"}
Jul 13 21:34:00.339621 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:00.339549    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:34:00.339621 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:00.339568    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:34:00.339910 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:00.339551    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:34:03.409996 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:03.409955    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:34:03.410525 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:03.410012    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:34:03.412802 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:03.412767    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") pod \"image-registry-79f645bcc9-drsxw\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") " pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:34:03.412802 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:03.412781    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc99873-ff7d-4ebe-80d8-3147d30440ca-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-6dsxq\" (UID: \"8cc99873-ff7d-4ebe-80d8-3147d30440ca\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:34:03.511354 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:03.511306    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:34:03.513980 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:03.513948    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418e5fc-2e14-430d-aee2-37c5eeae27bb-metrics-tls\") pod \"dns-default-xcd5s\" (UID: \"4418e5fc-2e14-430d-aee2-37c5eeae27bb\") " pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:34:03.644084 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:03.644041    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l6v89\""
Jul 13 21:34:03.644258 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:03.644100    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-khk5w\""
Jul 13 21:34:03.644258 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:03.644114    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rpb8r\""
Jul 13 21:34:03.651019 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:03.650981    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:34:03.651189 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:03.651048    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:34:03.651189 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:03.650984    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"
Jul 13 21:34:04.050280 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:04.050217    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xcd5s"]
Jul 13 21:34:04.053286 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:34:04.053194    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4418e5fc_2e14_430d_aee2_37c5eeae27bb.slice/crio-444ecf76c7c73864d47b660c39e783e0cbe140f86bab103cfd015635e73a8067 WatchSource:0}: Error finding container 444ecf76c7c73864d47b660c39e783e0cbe140f86bab103cfd015635e73a8067: Status 404 returned error can't find the container with id 444ecf76c7c73864d47b660c39e783e0cbe140f86bab103cfd015635e73a8067
Jul 13 21:34:04.066979 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:04.066945    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-6dsxq"]
Jul 13 21:34:04.070963 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:34:04.070918    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cc99873_ff7d_4ebe_80d8_3147d30440ca.slice/crio-bf0ce7f6d3fc2d98f72a7db6b943ebe7cf6b7a4d765cdd9b4ba6a40f1fe0adeb WatchSource:0}: Error finding container bf0ce7f6d3fc2d98f72a7db6b943ebe7cf6b7a4d765cdd9b4ba6a40f1fe0adeb: Status 404 returned error can't find the container with id bf0ce7f6d3fc2d98f72a7db6b943ebe7cf6b7a4d765cdd9b4ba6a40f1fe0adeb
Jul 13 21:34:04.094929 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:04.094896    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79f645bcc9-drsxw"]
Jul 13 21:34:04.097679 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:34:04.097632    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod860f54f5_378e_478c_a414_dc546c02c05b.slice/crio-c605e06173fef59c2244469f2f56ef551d5ea942242ad579ff0110a94e9b74fd WatchSource:0}: Error finding container c605e06173fef59c2244469f2f56ef551d5ea942242ad579ff0110a94e9b74fd: Status 404 returned error can't find the container with id c605e06173fef59c2244469f2f56ef551d5ea942242ad579ff0110a94e9b74fd
Jul 13 21:34:04.356594 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:04.356553    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw" event={"ID":"860f54f5-378e-478c-a414-dc546c02c05b","Type":"ContainerStarted","Data":"2c3022550ba945b378d6b517bae18d5e520a63bca598f1017c2b4da3871450cd"}
Jul 13 21:34:04.356800 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:04.356612    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw" event={"ID":"860f54f5-378e-478c-a414-dc546c02c05b","Type":"ContainerStarted","Data":"c605e06173fef59c2244469f2f56ef551d5ea942242ad579ff0110a94e9b74fd"}
Jul 13 21:34:04.356800 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:04.356639    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:34:04.357889 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:04.357867    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xcd5s" event={"ID":"4418e5fc-2e14-430d-aee2-37c5eeae27bb","Type":"ContainerStarted","Data":"444ecf76c7c73864d47b660c39e783e0cbe140f86bab103cfd015635e73a8067"}
Jul 13 21:34:04.359041 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:04.359017    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq" event={"ID":"8cc99873-ff7d-4ebe-80d8-3147d30440ca","Type":"ContainerStarted","Data":"bf0ce7f6d3fc2d98f72a7db6b943ebe7cf6b7a4d765cdd9b4ba6a40f1fe0adeb"}
Jul 13 21:34:04.360610 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:04.360590    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cnzqd" event={"ID":"70c39abc-b85c-4453-95ad-d9ca15d1b706","Type":"ContainerStarted","Data":"0c698edfb2984df144ce992a8a2f45f2e0f7d8698e79628caf0669bdac579d6b"}
Jul 13 21:34:04.376399 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:04.376351    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw" podStartSLOduration=263.376338251 podStartE2EDuration="4m23.376338251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:04.37522992 +0000 UTC m=+286.297278850" watchObservedRunningTime="2026-07-13 21:34:04.376338251 +0000 UTC m=+286.298387167"
Jul 13 21:34:05.367881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:05.367852    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cnzqd" event={"ID":"70c39abc-b85c-4453-95ad-d9ca15d1b706","Type":"ContainerStarted","Data":"4047d1ec58519d0436a6b2f66393a01c3aa454d62f675a33cb041e241de3d9a4"}
Jul 13 21:34:06.373600 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:06.373552    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq" event={"ID":"8cc99873-ff7d-4ebe-80d8-3147d30440ca","Type":"ContainerStarted","Data":"0a1fbff46ce33105bf979cb11a38052c1cb7e2229b2df3e7d44af2850933b88a"}
Jul 13 21:34:06.388852 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:06.388667    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-6dsxq" podStartSLOduration=261.10775441 podStartE2EDuration="4m22.388627535s" totalImagesPullingTime="1.280873125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:06.388156982 +0000 UTC m=+288.310205916" watchObservedRunningTime="2026-07-13 21:34:06.388627535 +0000 UTC m=+288.310676430"
Jul 13 21:34:07.378589 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:07.378559    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cnzqd" event={"ID":"70c39abc-b85c-4453-95ad-d9ca15d1b706","Type":"ContainerStarted","Data":"38fe20a4c3815350a512d46552b562ebeedde8f6746307f4599a0f7d5bc67d64"}
Jul 13 21:34:07.380568 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:07.380538    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xcd5s" event={"ID":"4418e5fc-2e14-430d-aee2-37c5eeae27bb","Type":"ContainerStarted","Data":"fc9c8600472611ed3bdff0c4caf2cd2062f2a80d581e782d6ba14646f8e4826d"}
Jul 13 21:34:07.380710 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:07.380586    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xcd5s" event={"ID":"4418e5fc-2e14-430d-aee2-37c5eeae27bb","Type":"ContainerStarted","Data":"d3fc996d221da62965f31b1762adaed91c9a46e9e1dbeb04ecd6a119657dbde2"}
Jul 13 21:34:07.397523 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:07.397443    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cnzqd" podStartSLOduration=17.188986522 podStartE2EDuration="24.397421445s" totalImagesPullingTime="7.208434923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:33:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:07.396296155 +0000 UTC m=+289.318345072" watchObservedRunningTime="2026-07-13 21:34:07.397421445 +0000 UTC m=+289.319470362"
Jul 13 21:34:07.411203 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:07.411148    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xcd5s" podStartSLOduration=251.26388907 podStartE2EDuration="4m14.411131919s" totalImagesPullingTime="3.147242849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:07.410710369 +0000 UTC m=+289.332759278" watchObservedRunningTime="2026-07-13 21:34:07.411131919 +0000 UTC m=+289.333180835"
Jul 13 21:34:08.384810 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:08.384777    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:34:10.264050 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:10.264009    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:34:10.266458 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:10.266434    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c267f5a-72ce-4e51-89ad-676469abda4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-h685t\" (UID: \"5c267f5a-72ce-4e51-89ad-676469abda4f\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:34:10.468091 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:34:10.468047    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fjzfw" podUID="4b89f48b-c81d-40dc-a85d-ea77e98bc1f5"
Jul 13 21:34:10.563257 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:10.563158    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"
Jul 13 21:34:10.690619 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:10.690420    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t"]
Jul 13 21:34:10.693288 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:34:10.693255    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c267f5a_72ce_4e51_89ad_676469abda4f.slice/crio-cac01c719b76135581c39583d0551986c0eae952ba386f7e5deae50f7ee50f02 WatchSource:0}: Error finding container cac01c719b76135581c39583d0551986c0eae952ba386f7e5deae50f7ee50f02: Status 404 returned error can't find the container with id cac01c719b76135581c39583d0551986c0eae952ba386f7e5deae50f7ee50f02
Jul 13 21:34:11.399815 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:11.399782    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t" event={"ID":"5c267f5a-72ce-4e51-89ad-676469abda4f","Type":"ContainerStarted","Data":"cac01c719b76135581c39583d0551986c0eae952ba386f7e5deae50f7ee50f02"}
Jul 13 21:34:13.215156 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:13.215116    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5"]
Jul 13 21:34:13.218674 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:13.218642    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5"
Jul 13 21:34:13.220858 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:13.220830    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 13 21:34:13.220979 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:13.220916    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-msllz\""
Jul 13 21:34:13.227335 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:13.227310    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5"]
Jul 13 21:34:13.287022 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:13.286983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/827c7efa-4d29-4f71-a846-7dee96d13cb0-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-x98w5\" (UID: \"827c7efa-4d29-4f71-a846-7dee96d13cb0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5"
Jul 13 21:34:13.387890 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:13.387847    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/827c7efa-4d29-4f71-a846-7dee96d13cb0-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-x98w5\" (UID: \"827c7efa-4d29-4f71-a846-7dee96d13cb0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5"
Jul 13 21:34:13.388108 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:34:13.388015    2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 13 21:34:13.388108 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:34:13.388107    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/827c7efa-4d29-4f71-a846-7dee96d13cb0-tls-certificates podName:827c7efa-4d29-4f71-a846-7dee96d13cb0 nodeName:}" failed. No retries permitted until 2026-07-13 21:34:13.88808588 +0000 UTC m=+295.810134780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/827c7efa-4d29-4f71-a846-7dee96d13cb0-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-x98w5" (UID: "827c7efa-4d29-4f71-a846-7dee96d13cb0") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 13 21:34:13.408610 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:13.408571    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t" event={"ID":"5c267f5a-72ce-4e51-89ad-676469abda4f","Type":"ContainerStarted","Data":"7f343128a5c22501452b36d575510848bc51d77484a7a35b04673ab0c4e06876"}
Jul 13 21:34:13.423797 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:13.423745    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-h685t" podStartSLOduration=33.3754193 podStartE2EDuration="35.423727648s" totalImagesPullingTime="2.048308348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:13.422856553 +0000 UTC m=+295.344905470" watchObservedRunningTime="2026-07-13 21:34:13.423727648 +0000 UTC m=+295.345776564"
Jul 13 21:34:13.891642 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:13.891601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/827c7efa-4d29-4f71-a846-7dee96d13cb0-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-x98w5\" (UID: \"827c7efa-4d29-4f71-a846-7dee96d13cb0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5"
Jul 13 21:34:13.894236 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:13.894200    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/827c7efa-4d29-4f71-a846-7dee96d13cb0-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-x98w5\" (UID: \"827c7efa-4d29-4f71-a846-7dee96d13cb0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5"
Jul 13 21:34:14.129697 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:14.129635    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5"
Jul 13 21:34:14.255552 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:14.255519    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5"]
Jul 13 21:34:14.258675 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:34:14.258628    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod827c7efa_4d29_4f71_a846_7dee96d13cb0.slice/crio-1fbcd44570945dffa10ce95543f16a0d30396699b066c12a63d3a4850bcbfe45 WatchSource:0}: Error finding container 1fbcd44570945dffa10ce95543f16a0d30396699b066c12a63d3a4850bcbfe45: Status 404 returned error can't find the container with id 1fbcd44570945dffa10ce95543f16a0d30396699b066c12a63d3a4850bcbfe45
Jul 13 21:34:14.413131 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:14.413087    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5" event={"ID":"827c7efa-4d29-4f71-a846-7dee96d13cb0","Type":"ContainerStarted","Data":"1fbcd44570945dffa10ce95543f16a0d30396699b066c12a63d3a4850bcbfe45"}
Jul 13 21:34:16.421301 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:16.421266    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5" event={"ID":"827c7efa-4d29-4f71-a846-7dee96d13cb0","Type":"ContainerStarted","Data":"cff627d990d4905a593faa51980c0d36742c84385ea31c630b51cd7bd604186a"}
Jul 13 21:34:16.421808 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:16.421489    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5"
Jul 13 21:34:16.426523 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:16.426497    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5"
Jul 13 21:34:16.437145 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:16.437094    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x98w5" podStartSLOduration=2.243599872 podStartE2EDuration="3.437075901s" totalImagesPullingTime="1.193476029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:34:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:16.435465289 +0000 UTC m=+298.357514206" watchObservedRunningTime="2026-07-13 21:34:16.437075901 +0000 UTC m=+298.359124858"
Jul 13 21:34:17.269064 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.269031    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-th5wg"]
Jul 13 21:34:17.272368 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.272353    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:17.275328 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.275300    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-r76zz\""
Jul 13 21:34:17.275465 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.275334    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 13 21:34:17.275465 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.275343    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 13 21:34:17.275465 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.275363    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 13 21:34:17.282070 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.282045    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-th5wg"]
Jul 13 21:34:17.319370 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.319334    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/018b60a2-9e31-4968-bca8-e96049846740-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:17.319370 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.319367    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/018b60a2-9e31-4968-bca8-e96049846740-metrics-client-ca\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:17.319582 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.319392    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/018b60a2-9e31-4968-bca8-e96049846740-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:17.319582 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.319488    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc4ks\" (UniqueName: \"kubernetes.io/projected/018b60a2-9e31-4968-bca8-e96049846740-kube-api-access-kc4ks\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:17.420533 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.420493    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc4ks\" (UniqueName: \"kubernetes.io/projected/018b60a2-9e31-4968-bca8-e96049846740-kube-api-access-kc4ks\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:17.420764 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.420558    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/018b60a2-9e31-4968-bca8-e96049846740-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:17.420764 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.420579    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/018b60a2-9e31-4968-bca8-e96049846740-metrics-client-ca\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:17.420764 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.420603    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/018b60a2-9e31-4968-bca8-e96049846740-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:17.420764 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:34:17.420739    2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 13 21:34:17.420995 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:34:17.420799    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/018b60a2-9e31-4968-bca8-e96049846740-prometheus-operator-tls podName:018b60a2-9e31-4968-bca8-e96049846740 nodeName:}" failed. No retries permitted until 2026-07-13 21:34:17.920779569 +0000 UTC m=+299.842828468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/018b60a2-9e31-4968-bca8-e96049846740-prometheus-operator-tls") pod "prometheus-operator-64785b6675-th5wg" (UID: "018b60a2-9e31-4968-bca8-e96049846740") : secret "prometheus-operator-tls" not found
Jul 13 21:34:17.421467 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.421442    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/018b60a2-9e31-4968-bca8-e96049846740-metrics-client-ca\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:17.423701 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.423674    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/018b60a2-9e31-4968-bca8-e96049846740-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:17.428381 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.428355    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc4ks\" (UniqueName: \"kubernetes.io/projected/018b60a2-9e31-4968-bca8-e96049846740-kube-api-access-kc4ks\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:17.924208 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:17.924171    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/018b60a2-9e31-4968-bca8-e96049846740-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:17.924395 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:34:17.924292    2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 13 21:34:17.924395 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:34:17.924344    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/018b60a2-9e31-4968-bca8-e96049846740-prometheus-operator-tls podName:018b60a2-9e31-4968-bca8-e96049846740 nodeName:}" failed. No retries permitted until 2026-07-13 21:34:18.924331535 +0000 UTC m=+300.846380431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/018b60a2-9e31-4968-bca8-e96049846740-prometheus-operator-tls") pod "prometheus-operator-64785b6675-th5wg" (UID: "018b60a2-9e31-4968-bca8-e96049846740") : secret "prometheus-operator-tls" not found
Jul 13 21:34:18.376295 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:18.376262    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:34:18.376545 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:18.376530    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:34:18.382911 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:18.382882    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 13 21:34:18.393871 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:18.393848    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xcd5s"
Jul 13 21:34:18.931712 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:18.931673    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/018b60a2-9e31-4968-bca8-e96049846740-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:18.934110 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:18.934088    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/018b60a2-9e31-4968-bca8-e96049846740-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-th5wg\" (UID: \"018b60a2-9e31-4968-bca8-e96049846740\") " pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:19.083125 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:19.083092    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-r76zz\""
Jul 13 21:34:19.091334 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:19.091307    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg"
Jul 13 21:34:19.223779 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:19.223705    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-th5wg"]
Jul 13 21:34:19.226747 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:34:19.226714    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod018b60a2_9e31_4968_bca8_e96049846740.slice/crio-286f5d6c6df5c31b9ba9f4cf51fcb394dd19cb02998284079a0fe2203445c17e WatchSource:0}: Error finding container 286f5d6c6df5c31b9ba9f4cf51fcb394dd19cb02998284079a0fe2203445c17e: Status 404 returned error can't find the container with id 286f5d6c6df5c31b9ba9f4cf51fcb394dd19cb02998284079a0fe2203445c17e
Jul 13 21:34:19.433198 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:19.433161    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg" event={"ID":"018b60a2-9e31-4968-bca8-e96049846740","Type":"ContainerStarted","Data":"286f5d6c6df5c31b9ba9f4cf51fcb394dd19cb02998284079a0fe2203445c17e"}
Jul 13 21:34:21.444670 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:21.444622    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg" event={"ID":"018b60a2-9e31-4968-bca8-e96049846740","Type":"ContainerStarted","Data":"f2b78ee64e72f6385fbd399b4c4bd5fc9b3e00382767d4de0cfac31596fa6657"}
Jul 13 21:34:21.445077 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:21.444695    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg" event={"ID":"018b60a2-9e31-4968-bca8-e96049846740","Type":"ContainerStarted","Data":"23ab85f227584a70e3be994768c0edb7f47076c9a9ec109db53cb3c801898842"}
Jul 13 21:34:21.462454 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:21.462402    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-th5wg" podStartSLOduration=3.162665341 podStartE2EDuration="4.462386719s" totalImagesPullingTime="1.299721378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:34:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:21.460930043 +0000 UTC m=+303.382978972" watchObservedRunningTime="2026-07-13 21:34:21.462386719 +0000 UTC m=+303.384435614"
Jul 13 21:34:22.467146 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:22.467114    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:34:23.615847 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.614114    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qnb5j"]
Jul 13 21:34:23.618179 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.618154    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.620554 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.620531    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 13 21:34:23.620711 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.620679    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 13 21:34:23.620979 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.620963    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 13 21:34:23.621641 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.621469    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vvh8d\""
Jul 13 21:34:23.626614 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.626594    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-9rg85"]
Jul 13 21:34:23.630357 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.630333    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.632498 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.632480    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 13 21:34:23.632570 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.632536    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-sw2nt\""
Jul 13 21:34:23.632821 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.632801    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 13 21:34:23.633242 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.633225    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 13 21:34:23.642266 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.642238    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-9rg85"]
Jul 13 21:34:23.655625 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.655597    2564 patch_prober.go:28] interesting pod/image-registry-79f645bcc9-drsxw container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 13 21:34:23.655761 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.655643    2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw" podUID="860f54f5-378e-478c-a414-dc546c02c05b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 21:34:23.669206 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669180    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-wtmp\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.669358 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669213    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23529af6-bf21-404d-bac0-4674d7f26383-metrics-client-ca\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.669358 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669232    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gr8z\" (UniqueName: \"kubernetes.io/projected/ec8e284a-cc0b-4127-b7f0-b1475beae905-kube-api-access-6gr8z\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.669358 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669256    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-accelerators-collector-config\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.669358 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669274    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23529af6-bf21-404d-bac0-4674d7f26383-sys\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.669358 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669319    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.669542 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669395    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm999\" (UniqueName: \"kubernetes.io/projected/23529af6-bf21-404d-bac0-4674d7f26383-kube-api-access-sm999\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.669542 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669442    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec8e284a-cc0b-4127-b7f0-b1475beae905-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.669542 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669466    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ec8e284a-cc0b-4127-b7f0-b1475beae905-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.669542 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669487    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-textfile\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.669542 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-tls\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.669728 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669560    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ec8e284a-cc0b-4127-b7f0-b1475beae905-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.669728 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669583    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/23529af6-bf21-404d-bac0-4674d7f26383-root\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.669728 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669616    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec8e284a-cc0b-4127-b7f0-b1475beae905-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.669728 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.669639    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec8e284a-cc0b-4127-b7f0-b1475beae905-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.770424 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770389    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ec8e284a-cc0b-4127-b7f0-b1475beae905-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.770424 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-textfile\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.770703 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770444    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-tls\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.770703 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770486    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ec8e284a-cc0b-4127-b7f0-b1475beae905-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.770703 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770623    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/23529af6-bf21-404d-bac0-4674d7f26383-root\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.770703 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770699    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec8e284a-cc0b-4127-b7f0-b1475beae905-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.770903 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770730    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/23529af6-bf21-404d-bac0-4674d7f26383-root\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.770903 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770743    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec8e284a-cc0b-4127-b7f0-b1475beae905-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.770903 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-wtmp\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.770903 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23529af6-bf21-404d-bac0-4674d7f26383-metrics-client-ca\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.770903 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770835    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-textfile\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.770903 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770856    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gr8z\" (UniqueName: \"kubernetes.io/projected/ec8e284a-cc0b-4127-b7f0-b1475beae905-kube-api-access-6gr8z\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.770903 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-accelerators-collector-config\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.771227 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770936    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23529af6-bf21-404d-bac0-4674d7f26383-sys\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.771227 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770960    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.771227 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770977    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-wtmp\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.771227 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.770990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm999\" (UniqueName: \"kubernetes.io/projected/23529af6-bf21-404d-bac0-4674d7f26383-kube-api-access-sm999\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.771227 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.771014    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ec8e284a-cc0b-4127-b7f0-b1475beae905-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.771227 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.771027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec8e284a-cc0b-4127-b7f0-b1475beae905-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.771511 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.771273    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ec8e284a-cc0b-4127-b7f0-b1475beae905-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.771511 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.771448    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/23529af6-bf21-404d-bac0-4674d7f26383-metrics-client-ca\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.771624 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.771528    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/23529af6-bf21-404d-bac0-4674d7f26383-sys\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.771898 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.771869    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-accelerators-collector-config\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.771986 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.771965    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec8e284a-cc0b-4127-b7f0-b1475beae905-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.773354 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.773324    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-tls\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.773486 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.773407    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec8e284a-cc0b-4127-b7f0-b1475beae905-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.773582 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.773563    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec8e284a-cc0b-4127-b7f0-b1475beae905-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.773938 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.773917    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/23529af6-bf21-404d-bac0-4674d7f26383-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.778334 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.778310    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm999\" (UniqueName: \"kubernetes.io/projected/23529af6-bf21-404d-bac0-4674d7f26383-kube-api-access-sm999\") pod \"node-exporter-qnb5j\" (UID: \"23529af6-bf21-404d-bac0-4674d7f26383\") " pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.778492 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.778473    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gr8z\" (UniqueName: \"kubernetes.io/projected/ec8e284a-cc0b-4127-b7f0-b1475beae905-kube-api-access-6gr8z\") pod \"kube-state-metrics-785778ff74-9rg85\" (UID: \"ec8e284a-cc0b-4127-b7f0-b1475beae905\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.929502 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.929413    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qnb5j"
Jul 13 21:34:23.939446 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.939415    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85"
Jul 13 21:34:23.950788 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:23.950766    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:34:24.093324 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.093247    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-9rg85"]
Jul 13 21:34:24.098068 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:34:24.098039    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec8e284a_cc0b_4127_b7f0_b1475beae905.slice/crio-0d7a2f0edfbae6be2ceccb50b41f196c5475e18d194e31f4432f255836316b22 WatchSource:0}: Error finding container 0d7a2f0edfbae6be2ceccb50b41f196c5475e18d194e31f4432f255836316b22: Status 404 returned error can't find the container with id 0d7a2f0edfbae6be2ceccb50b41f196c5475e18d194e31f4432f255836316b22
Jul 13 21:34:24.455927 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.455892    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85" event={"ID":"ec8e284a-cc0b-4127-b7f0-b1475beae905","Type":"ContainerStarted","Data":"0d7a2f0edfbae6be2ceccb50b41f196c5475e18d194e31f4432f255836316b22"}
Jul 13 21:34:24.457041 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.457005    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qnb5j" event={"ID":"23529af6-bf21-404d-bac0-4674d7f26383","Type":"ContainerStarted","Data":"2f1237db1a3322bea396791d5b3c81c473fa2380335aeecd0ec8d74de10d3a0b"}
Jul 13 21:34:24.674985 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.674829    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:34:24.679117 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.679091    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.683786 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.681785    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 13 21:34:24.683786 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.682090    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 13 21:34:24.683786 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.682296    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 13 21:34:24.683786 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.682329    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 13 21:34:24.683786 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.682530    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 13 21:34:24.683786 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.682539    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 13 21:34:24.683786 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.682813    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 13 21:34:24.683786 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.683081    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 13 21:34:24.683786 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.683412    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-r5bjd\""
Jul 13 21:34:24.686683 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.684844    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 13 21:34:24.699138 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.699114    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:34:24.780083 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.779874    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a18a26d9-75ab-4d36-b619-84b9e92a5721-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.780083 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.779922    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a18a26d9-75ab-4d36-b619-84b9e92a5721-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.780083 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.779962    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a18a26d9-75ab-4d36-b619-84b9e92a5721-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.780083 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.779984    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a18a26d9-75ab-4d36-b619-84b9e92a5721-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.780083 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.780009    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4swsr\" (UniqueName: \"kubernetes.io/projected/a18a26d9-75ab-4d36-b619-84b9e92a5721-kube-api-access-4swsr\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.780549 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.780103    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.780549 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.780163    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-config-volume\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.780549 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.780198    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a18a26d9-75ab-4d36-b619-84b9e92a5721-config-out\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.780549 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.780238    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.780549 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.780268    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.780549 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.780292    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-web-config\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.780549 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.780319    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.780549 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.780376    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.881166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a18a26d9-75ab-4d36-b619-84b9e92a5721-config-out\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.881224    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.881253    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.881278    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-web-config\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.881308    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.881354    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.881391    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a18a26d9-75ab-4d36-b619-84b9e92a5721-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.881425    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a18a26d9-75ab-4d36-b619-84b9e92a5721-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.881460    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a18a26d9-75ab-4d36-b619-84b9e92a5721-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.881484    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a18a26d9-75ab-4d36-b619-84b9e92a5721-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.881508    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4swsr\" (UniqueName: \"kubernetes.io/projected/a18a26d9-75ab-4d36-b619-84b9e92a5721-kube-api-access-4swsr\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.881551    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.881594    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-config-volume\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.881719 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:34:24.881693    2564 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 13 21:34:24.882534 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:34:24.881758    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-main-tls podName:a18a26d9-75ab-4d36-b619-84b9e92a5721 nodeName:}" failed. No retries permitted until 2026-07-13 21:34:25.381739146 +0000 UTC m=+307.303788057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721") : secret "alertmanager-main-tls" not found
Jul 13 21:34:24.883032 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.883005    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a18a26d9-75ab-4d36-b619-84b9e92a5721-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.884364 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.883873    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a18a26d9-75ab-4d36-b619-84b9e92a5721-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.886948 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.886922    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a18a26d9-75ab-4d36-b619-84b9e92a5721-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.888028 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.887986    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a18a26d9-75ab-4d36-b619-84b9e92a5721-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.892803 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.892743    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.893306 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.893130    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4swsr\" (UniqueName: \"kubernetes.io/projected/a18a26d9-75ab-4d36-b619-84b9e92a5721-kube-api-access-4swsr\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.893306 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.893267    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.893869 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.893553    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-web-config\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.893869 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.893809    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a18a26d9-75ab-4d36-b619-84b9e92a5721-config-out\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.893869 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.893834    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-config-volume\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.894454 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.894419    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:24.896847 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:24.896807    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:25.373122 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:25.373085    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:34:25.387391 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:25.387356    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:25.390017 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:25.389993    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:25.468902 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:25.468812    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qnb5j" event={"ID":"23529af6-bf21-404d-bac0-4674d7f26383","Type":"ContainerStarted","Data":"22950dacc0bbea8ffa001721bf21b6fbc8105ba400f7ba5045702213f7578303"}
Jul 13 21:34:25.596405 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:25.596372    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:34:25.766238 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:25.766200    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:34:25.769505 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:34:25.769481    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18a26d9_75ab_4d36_b619_84b9e92a5721.slice/crio-9d4169b68075f35a170d8b418d348114b20fda69696c90d25016ee06ab9ddb70 WatchSource:0}: Error finding container 9d4169b68075f35a170d8b418d348114b20fda69696c90d25016ee06ab9ddb70: Status 404 returned error can't find the container with id 9d4169b68075f35a170d8b418d348114b20fda69696c90d25016ee06ab9ddb70
Jul 13 21:34:26.473731 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:26.473701    2564 generic.go:356] "Generic (PLEG): container finished" podID="23529af6-bf21-404d-bac0-4674d7f26383" containerID="22950dacc0bbea8ffa001721bf21b6fbc8105ba400f7ba5045702213f7578303" exitCode=0
Jul 13 21:34:26.473939 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:26.473785    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qnb5j" event={"ID":"23529af6-bf21-404d-bac0-4674d7f26383","Type":"ContainerDied","Data":"22950dacc0bbea8ffa001721bf21b6fbc8105ba400f7ba5045702213f7578303"}
Jul 13 21:34:26.475261 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:26.475223    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerStarted","Data":"9d4169b68075f35a170d8b418d348114b20fda69696c90d25016ee06ab9ddb70"}
Jul 13 21:34:26.477683 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:26.477547    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85" event={"ID":"ec8e284a-cc0b-4127-b7f0-b1475beae905","Type":"ContainerStarted","Data":"9f702266c07cef1db94a23c5b78329ebbc0b0fec84565c22989344d87a595de9"}
Jul 13 21:34:26.477683 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:26.477600    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85" event={"ID":"ec8e284a-cc0b-4127-b7f0-b1475beae905","Type":"ContainerStarted","Data":"493cb8c57119548880c07e46d7498cd6803d4281ad0cfe9828907a38f9151e94"}
Jul 13 21:34:26.477683 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:26.477627    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85" event={"ID":"ec8e284a-cc0b-4127-b7f0-b1475beae905","Type":"ContainerStarted","Data":"ff4df56a5c8537b31bffc54abc875389aadfe238a297eb06db2014ca59d55889"}
Jul 13 21:34:26.508818 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:26.508769    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-9rg85" podStartSLOduration=1.954170547 podStartE2EDuration="3.508750082s" totalImagesPullingTime="1.554579535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:26.507160831 +0000 UTC m=+308.429209749" watchObservedRunningTime="2026-07-13 21:34:26.508750082 +0000 UTC m=+308.430798998"
Jul 13 21:34:27.483144 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:27.483101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qnb5j" event={"ID":"23529af6-bf21-404d-bac0-4674d7f26383","Type":"ContainerStarted","Data":"fe1d89a4e14ba5973ecd0f5dbe2579e1a5b5277d7ec878f91103f48c698fc163"}
Jul 13 21:34:27.483607 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:27.483165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qnb5j" event={"ID":"23529af6-bf21-404d-bac0-4674d7f26383","Type":"ContainerStarted","Data":"879cbb9ad655f66b02cd1c9b8e466701e79a29fedbc0d47a1a98dd213bfa902b"}
Jul 13 21:34:27.484560 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:27.484538    2564 generic.go:356] "Generic (PLEG): container finished" podID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerID="04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f" exitCode=0
Jul 13 21:34:27.484671 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:27.484629    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerDied","Data":"04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f"}
Jul 13 21:34:27.503286 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:27.503231    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qnb5j" podStartSLOduration=3.181600274 podStartE2EDuration="4.5032146s" totalImagesPullingTime="1.247925897s" totalInitContainerRuntime="73.688429ms" isStatefulPod=true podCreationTimestamp="2026-07-13 21:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:27.501773135 +0000 UTC m=+309.423822068" watchObservedRunningTime="2026-07-13 21:34:27.5032146 +0000 UTC m=+309.425263516"
Jul 13 21:34:29.275336 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:29.275310    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79f645bcc9-drsxw"]
Jul 13 21:34:29.497483 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:29.497444    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerStarted","Data":"9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f"}
Jul 13 21:34:29.497622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:29.497513    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerStarted","Data":"539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617"}
Jul 13 21:34:29.497622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:29.497539    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerStarted","Data":"6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db"}
Jul 13 21:34:29.497622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:29.497563    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerStarted","Data":"1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111"}
Jul 13 21:34:29.497622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:29.497585    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerStarted","Data":"3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9"}
Jul 13 21:34:30.504423 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:30.504387    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerStarted","Data":"8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6"}
Jul 13 21:34:30.530694 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:30.530628    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.810999981 podStartE2EDuration="6.530609379s" totalImagesPullingTime="3.703463017s" totalInitContainerRuntime="16.146381ms" isStatefulPod=false podCreationTimestamp="2026-07-13 21:34:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:30.528593881 +0000 UTC m=+312.450642821" watchObservedRunningTime="2026-07-13 21:34:30.530609379 +0000 UTC m=+312.452658296"
Jul 13 21:34:32.075890 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.075856    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67f89b8757-lngxw"]
Jul 13 21:34:32.081067 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.081047    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.084144 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.084116    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2rzhp\""
Jul 13 21:34:32.084283 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.084120    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 13 21:34:32.084283 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.084168    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 13 21:34:32.084283 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.084216    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 13 21:34:32.084283 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.084249    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 13 21:34:32.084283 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.084257    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 13 21:34:32.084570 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.084362    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 13 21:34:32.084570 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.084479    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 13 21:34:32.088456 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.088432    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67f89b8757-lngxw"]
Jul 13 21:34:32.152240 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.152207    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-config\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.152240 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.152248    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-serving-cert\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.152464 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.152318    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b69qf\" (UniqueName: \"kubernetes.io/projected/29768d5e-b3f4-4820-a7ab-a9a2c621727a-kube-api-access-b69qf\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.152464 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.152360    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-oauth-serving-cert\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.152464 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.152412    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-service-ca\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.152464 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.152439    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-oauth-config\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.253836 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.253800    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b69qf\" (UniqueName: \"kubernetes.io/projected/29768d5e-b3f4-4820-a7ab-a9a2c621727a-kube-api-access-b69qf\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.254013 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.253846    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-oauth-serving-cert\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.254013 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.253991    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-service-ca\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.254120 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.254043    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-oauth-config\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.254120 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.254066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-config\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.254224 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.254195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-serving-cert\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.254604 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.254575    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-oauth-serving-cert\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.254731 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.254638    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-service-ca\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.254782 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.254742    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-config\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.256370 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.256352    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-oauth-config\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.256736 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.256717    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-serving-cert\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.261412 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.261386    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b69qf\" (UniqueName: \"kubernetes.io/projected/29768d5e-b3f4-4820-a7ab-a9a2c621727a-kube-api-access-b69qf\") pod \"console-67f89b8757-lngxw\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") " pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.392185 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.392087    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:32.526159 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:32.526126    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67f89b8757-lngxw"]
Jul 13 21:34:32.529543 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:34:32.529514    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29768d5e_b3f4_4820_a7ab_a9a2c621727a.slice/crio-0a9fe6c328c83dd95cbd5ce26d924f2bd5342b1c710b9401a2c07463946225cb WatchSource:0}: Error finding container 0a9fe6c328c83dd95cbd5ce26d924f2bd5342b1c710b9401a2c07463946225cb: Status 404 returned error can't find the container with id 0a9fe6c328c83dd95cbd5ce26d924f2bd5342b1c710b9401a2c07463946225cb
Jul 13 21:34:33.523989 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:33.523954    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f89b8757-lngxw" event={"ID":"29768d5e-b3f4-4820-a7ab-a9a2c621727a","Type":"ContainerStarted","Data":"0a9fe6c328c83dd95cbd5ce26d924f2bd5342b1c710b9401a2c07463946225cb"}
Jul 13 21:34:36.540702 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:36.540666    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f89b8757-lngxw" event={"ID":"29768d5e-b3f4-4820-a7ab-a9a2c621727a","Type":"ContainerStarted","Data":"24cd68b9e057574b4c7a9879debf82e483423fdd4eed340e850a05696634a37d"}
Jul 13 21:34:36.557108 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:36.557057    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67f89b8757-lngxw" podStartSLOduration=0.708849062 podStartE2EDuration="4.557041074s" totalImagesPullingTime="3.848192012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:34:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:36.55561415 +0000 UTC m=+318.477663079" watchObservedRunningTime="2026-07-13 21:34:36.557041074 +0000 UTC m=+318.479089987"
Jul 13 21:34:41.487606 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.487568    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-987dd7958-z7kbl"]
Jul 13 21:34:41.490739 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.490720    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.505920 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.505875    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-987dd7958-z7kbl"]
Jul 13 21:34:41.506073 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.506052    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 13 21:34:41.643000 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.642960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v25jd\" (UniqueName: \"kubernetes.io/projected/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-kube-api-access-v25jd\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.643000 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.643007    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-service-ca\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.643221 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.643036    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-config\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.643221 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.643085    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-trusted-ca-bundle\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.643221 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.643152    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-serving-cert\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.643221 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.643204    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-oauth-config\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.643221 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.643220    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-oauth-serving-cert\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.744106 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.744026    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-oauth-config\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.744106 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.744064    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-oauth-serving-cert\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.744106 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.744093    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v25jd\" (UniqueName: \"kubernetes.io/projected/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-kube-api-access-v25jd\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.744345 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.744115    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-service-ca\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.744345 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.744135    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-config\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.744345 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.744259    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-trusted-ca-bundle\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.744345 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.744303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-serving-cert\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.744955 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.744930    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-config\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.745081 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.744963    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-service-ca\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.745081 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.744979    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-oauth-serving-cert\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.745179 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.745161    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-trusted-ca-bundle\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.746623 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.746597    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-oauth-config\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.746843 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.746820    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-serving-cert\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.752367 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.752342    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v25jd\" (UniqueName: \"kubernetes.io/projected/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-kube-api-access-v25jd\") pod \"console-987dd7958-z7kbl\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") " pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.804276 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.804240    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:41.941540 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:41.941447    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-987dd7958-z7kbl"]
Jul 13 21:34:41.944310 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:34:41.944282    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff9f1ba_d7c1_4012_a917_88d586ff29ac.slice/crio-2114e9e2c8de8179de76c0c2a25c4176c8f41f0924c6877122345a7095b9bb3f WatchSource:0}: Error finding container 2114e9e2c8de8179de76c0c2a25c4176c8f41f0924c6877122345a7095b9bb3f: Status 404 returned error can't find the container with id 2114e9e2c8de8179de76c0c2a25c4176c8f41f0924c6877122345a7095b9bb3f
Jul 13 21:34:42.392851 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:42.392810    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:42.393058 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:42.392893    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:42.397559 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:42.397535    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:42.566598 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:42.566552    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-987dd7958-z7kbl" event={"ID":"7ff9f1ba-d7c1-4012-a917-88d586ff29ac","Type":"ContainerStarted","Data":"ae7558fc07237c63aeaeb82606c7919d10424576429f28eee0fa43984ed69ac9"}
Jul 13 21:34:42.567020 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:42.566618    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-987dd7958-z7kbl" event={"ID":"7ff9f1ba-d7c1-4012-a917-88d586ff29ac","Type":"ContainerStarted","Data":"2114e9e2c8de8179de76c0c2a25c4176c8f41f0924c6877122345a7095b9bb3f"}
Jul 13 21:34:42.570441 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:42.570422    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:34:42.584542 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:42.584481    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-987dd7958-z7kbl" podStartSLOduration=1.5844620919999999 podStartE2EDuration="1.584462092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:34:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:34:42.58307119 +0000 UTC m=+324.505120106" watchObservedRunningTime="2026-07-13 21:34:42.584462092 +0000 UTC m=+324.506511009"
Jul 13 21:34:51.804389 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:51.804346    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:51.804903 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:51.804411    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:51.809042 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:51.809021    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:52.613154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:52.613122    2564 generic.go:356] "Generic (PLEG): container finished" podID="675ce41d-5f47-4029-aaf7-504ad5f2fc06" containerID="332f7f3c842d5ba3e0fbd55f195c6964281015ce55c408c980df5f4348b20e2d" exitCode=0
Jul 13 21:34:52.613344 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:52.613200    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz" event={"ID":"675ce41d-5f47-4029-aaf7-504ad5f2fc06","Type":"ContainerDied","Data":"332f7f3c842d5ba3e0fbd55f195c6964281015ce55c408c980df5f4348b20e2d"}
Jul 13 21:34:52.613791 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:52.613744    2564 scope.go:117] "RemoveContainer" containerID="332f7f3c842d5ba3e0fbd55f195c6964281015ce55c408c980df5f4348b20e2d"
Jul 13 21:34:52.617688 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:52.617664    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:34:52.678143 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:52.678114    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67f89b8757-lngxw"]
Jul 13 21:34:53.619402 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:53.619366    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-v9sjz" event={"ID":"675ce41d-5f47-4029-aaf7-504ad5f2fc06","Type":"ContainerStarted","Data":"71170b5d9e4a714be384a034fd3212cdc5eab193989805176fa09471e8d8b6fc"}
Jul 13 21:34:54.297319 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.297280    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw" podUID="860f54f5-378e-478c-a414-dc546c02c05b" containerName="registry" containerID="cri-o://2c3022550ba945b378d6b517bae18d5e520a63bca598f1017c2b4da3871450cd" gracePeriod=30
Jul 13 21:34:54.545882 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.545858    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:34:54.624872 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.624784    2564 generic.go:356] "Generic (PLEG): container finished" podID="860f54f5-378e-478c-a414-dc546c02c05b" containerID="2c3022550ba945b378d6b517bae18d5e520a63bca598f1017c2b4da3871450cd" exitCode=0
Jul 13 21:34:54.624872 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.624832    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw"
Jul 13 21:34:54.625278 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.624871    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw" event={"ID":"860f54f5-378e-478c-a414-dc546c02c05b","Type":"ContainerDied","Data":"2c3022550ba945b378d6b517bae18d5e520a63bca598f1017c2b4da3871450cd"}
Jul 13 21:34:54.625278 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.624925    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79f645bcc9-drsxw" event={"ID":"860f54f5-378e-478c-a414-dc546c02c05b","Type":"ContainerDied","Data":"c605e06173fef59c2244469f2f56ef551d5ea942242ad579ff0110a94e9b74fd"}
Jul 13 21:34:54.625278 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.624948    2564 scope.go:117] "RemoveContainer" containerID="2c3022550ba945b378d6b517bae18d5e520a63bca598f1017c2b4da3871450cd"
Jul 13 21:34:54.634573 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.634555    2564 scope.go:117] "RemoveContainer" containerID="2c3022550ba945b378d6b517bae18d5e520a63bca598f1017c2b4da3871450cd"
Jul 13 21:34:54.634875 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:34:54.634854    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c3022550ba945b378d6b517bae18d5e520a63bca598f1017c2b4da3871450cd\": container with ID starting with 2c3022550ba945b378d6b517bae18d5e520a63bca598f1017c2b4da3871450cd not found: ID does not exist" containerID="2c3022550ba945b378d6b517bae18d5e520a63bca598f1017c2b4da3871450cd"
Jul 13 21:34:54.634948 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.634889    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c3022550ba945b378d6b517bae18d5e520a63bca598f1017c2b4da3871450cd"} err="failed to get container status \"2c3022550ba945b378d6b517bae18d5e520a63bca598f1017c2b4da3871450cd\": rpc error: code = NotFound desc = could not find container \"2c3022550ba945b378d6b517bae18d5e520a63bca598f1017c2b4da3871450cd\": container with ID starting with 2c3022550ba945b378d6b517bae18d5e520a63bca598f1017c2b4da3871450cd not found: ID does not exist"
Jul 13 21:34:54.662791 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.662757    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/860f54f5-378e-478c-a414-dc546c02c05b-installation-pull-secrets\") pod \"860f54f5-378e-478c-a414-dc546c02c05b\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") "
Jul 13 21:34:54.662912 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.662800    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/860f54f5-378e-478c-a414-dc546c02c05b-image-registry-private-configuration\") pod \"860f54f5-378e-478c-a414-dc546c02c05b\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") "
Jul 13 21:34:54.662912 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.662876    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-bound-sa-token\") pod \"860f54f5-378e-478c-a414-dc546c02c05b\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") "
Jul 13 21:34:54.663032 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.662939    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/860f54f5-378e-478c-a414-dc546c02c05b-trusted-ca\") pod \"860f54f5-378e-478c-a414-dc546c02c05b\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") "
Jul 13 21:34:54.663032 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.662964    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") pod \"860f54f5-378e-478c-a414-dc546c02c05b\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") "
Jul 13 21:34:54.663032 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.662997    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr8x6\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-kube-api-access-hr8x6\") pod \"860f54f5-378e-478c-a414-dc546c02c05b\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") "
Jul 13 21:34:54.663174 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.663086    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/860f54f5-378e-478c-a414-dc546c02c05b-ca-trust-extracted\") pod \"860f54f5-378e-478c-a414-dc546c02c05b\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") "
Jul 13 21:34:54.663174 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.663128    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/860f54f5-378e-478c-a414-dc546c02c05b-registry-certificates\") pod \"860f54f5-378e-478c-a414-dc546c02c05b\" (UID: \"860f54f5-378e-478c-a414-dc546c02c05b\") "
Jul 13 21:34:54.663845 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.663384    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/860f54f5-378e-478c-a414-dc546c02c05b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "860f54f5-378e-478c-a414-dc546c02c05b" (UID: "860f54f5-378e-478c-a414-dc546c02c05b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:34:54.663845 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.663537    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/860f54f5-378e-478c-a414-dc546c02c05b-trusted-ca\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:34:54.663845 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.663729    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/860f54f5-378e-478c-a414-dc546c02c05b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "860f54f5-378e-478c-a414-dc546c02c05b" (UID: "860f54f5-378e-478c-a414-dc546c02c05b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:34:54.665469 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.665398    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "860f54f5-378e-478c-a414-dc546c02c05b" (UID: "860f54f5-378e-478c-a414-dc546c02c05b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:34:54.665620 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.665595    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "860f54f5-378e-478c-a414-dc546c02c05b" (UID: "860f54f5-378e-478c-a414-dc546c02c05b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:34:54.665724 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.665619    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/860f54f5-378e-478c-a414-dc546c02c05b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "860f54f5-378e-478c-a414-dc546c02c05b" (UID: "860f54f5-378e-478c-a414-dc546c02c05b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:34:54.665724 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.665702    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-kube-api-access-hr8x6" (OuterVolumeSpecName: "kube-api-access-hr8x6") pod "860f54f5-378e-478c-a414-dc546c02c05b" (UID: "860f54f5-378e-478c-a414-dc546c02c05b"). InnerVolumeSpecName "kube-api-access-hr8x6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:34:54.665808 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.665710    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/860f54f5-378e-478c-a414-dc546c02c05b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "860f54f5-378e-478c-a414-dc546c02c05b" (UID: "860f54f5-378e-478c-a414-dc546c02c05b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:34:54.674908 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.674875    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/860f54f5-378e-478c-a414-dc546c02c05b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "860f54f5-378e-478c-a414-dc546c02c05b" (UID: "860f54f5-378e-478c-a414-dc546c02c05b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:34:54.764038 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.764002    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-bound-sa-token\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:34:54.764038 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.764033    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-registry-tls\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:34:54.764038 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.764043    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr8x6\" (UniqueName: \"kubernetes.io/projected/860f54f5-378e-478c-a414-dc546c02c05b-kube-api-access-hr8x6\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:34:54.764271 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.764053    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/860f54f5-378e-478c-a414-dc546c02c05b-ca-trust-extracted\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:34:54.764271 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.764061    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/860f54f5-378e-478c-a414-dc546c02c05b-registry-certificates\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:34:54.764271 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.764070    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/860f54f5-378e-478c-a414-dc546c02c05b-installation-pull-secrets\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:34:54.764271 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.764079    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/860f54f5-378e-478c-a414-dc546c02c05b-image-registry-private-configuration\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:34:54.946580 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.946547    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79f645bcc9-drsxw"]
Jul 13 21:34:54.950350 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:54.950325    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-79f645bcc9-drsxw"]
Jul 13 21:34:56.471881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:34:56.471838    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="860f54f5-378e-478c-a414-dc546c02c05b" path="/var/lib/kubelet/pods/860f54f5-378e-478c-a414-dc546c02c05b/volumes"
Jul 13 21:35:07.677989 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:07.677957    2564 generic.go:356] "Generic (PLEG): container finished" podID="6ca58477-dcef-4741-9227-4842c003a667" containerID="edb7d52811cc54c6a4c89338637ff727eb6987117dd95a4ccb99ddb8178a84b9" exitCode=0
Jul 13 21:35:07.678355 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:07.678001    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx" event={"ID":"6ca58477-dcef-4741-9227-4842c003a667","Type":"ContainerDied","Data":"edb7d52811cc54c6a4c89338637ff727eb6987117dd95a4ccb99ddb8178a84b9"}
Jul 13 21:35:07.678355 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:07.678290    2564 scope.go:117] "RemoveContainer" containerID="edb7d52811cc54c6a4c89338637ff727eb6987117dd95a4ccb99ddb8178a84b9"
Jul 13 21:35:08.684633 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:08.684600    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jwdmx" event={"ID":"6ca58477-dcef-4741-9227-4842c003a667","Type":"ContainerStarted","Data":"913f7d9d5b76f8f04cfc16815b5f24866d9fc27c3ad2714b9fc3834bbd785ab4"}
Jul 13 21:35:17.703625 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:17.703365    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67f89b8757-lngxw" podUID="29768d5e-b3f4-4820-a7ab-a9a2c621727a" containerName="console" containerID="cri-o://24cd68b9e057574b4c7a9879debf82e483423fdd4eed340e850a05696634a37d" gracePeriod=15
Jul 13 21:35:17.982121 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:17.982086    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:35:18.071140 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.069948    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-serving-cert\") pod \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") "
Jul 13 21:35:18.071140 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.070011    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-oauth-config\") pod \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") "
Jul 13 21:35:18.071140 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.070049    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b69qf\" (UniqueName: \"kubernetes.io/projected/29768d5e-b3f4-4820-a7ab-a9a2c621727a-kube-api-access-b69qf\") pod \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") "
Jul 13 21:35:18.071140 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.070113    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-config\") pod \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") "
Jul 13 21:35:18.071140 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.070143    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-oauth-serving-cert\") pod \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") "
Jul 13 21:35:18.071140 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.070174    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-service-ca\") pod \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\" (UID: \"29768d5e-b3f4-4820-a7ab-a9a2c621727a\") "
Jul 13 21:35:18.071140 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.070829    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-service-ca" (OuterVolumeSpecName: "service-ca") pod "29768d5e-b3f4-4820-a7ab-a9a2c621727a" (UID: "29768d5e-b3f4-4820-a7ab-a9a2c621727a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:35:18.072633 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.072393    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-config" (OuterVolumeSpecName: "console-config") pod "29768d5e-b3f4-4820-a7ab-a9a2c621727a" (UID: "29768d5e-b3f4-4820-a7ab-a9a2c621727a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:35:18.073165 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.072900    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "29768d5e-b3f4-4820-a7ab-a9a2c621727a" (UID: "29768d5e-b3f4-4820-a7ab-a9a2c621727a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:35:18.075627 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.075599    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "29768d5e-b3f4-4820-a7ab-a9a2c621727a" (UID: "29768d5e-b3f4-4820-a7ab-a9a2c621727a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:35:18.075846 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.075819    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "29768d5e-b3f4-4820-a7ab-a9a2c621727a" (UID: "29768d5e-b3f4-4820-a7ab-a9a2c621727a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:35:18.075846 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.075833    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29768d5e-b3f4-4820-a7ab-a9a2c621727a-kube-api-access-b69qf" (OuterVolumeSpecName: "kube-api-access-b69qf") pod "29768d5e-b3f4-4820-a7ab-a9a2c621727a" (UID: "29768d5e-b3f4-4820-a7ab-a9a2c621727a"). InnerVolumeSpecName "kube-api-access-b69qf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:35:18.171781 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.171641    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-config\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:18.171781 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.171692    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-oauth-serving-cert\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:18.171781 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.171708    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/29768d5e-b3f4-4820-a7ab-a9a2c621727a-service-ca\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:18.171781 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.171723    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-serving-cert\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:18.171781 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.171736    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/29768d5e-b3f4-4820-a7ab-a9a2c621727a-console-oauth-config\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:18.171781 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.171750    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b69qf\" (UniqueName: \"kubernetes.io/projected/29768d5e-b3f4-4820-a7ab-a9a2c621727a-kube-api-access-b69qf\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:18.727174 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.727085    2564 generic.go:356] "Generic (PLEG): container finished" podID="29768d5e-b3f4-4820-a7ab-a9a2c621727a" containerID="24cd68b9e057574b4c7a9879debf82e483423fdd4eed340e850a05696634a37d" exitCode=0
Jul 13 21:35:18.727174 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.727151    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f89b8757-lngxw"
Jul 13 21:35:18.727590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.727174    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f89b8757-lngxw" event={"ID":"29768d5e-b3f4-4820-a7ab-a9a2c621727a","Type":"ContainerDied","Data":"24cd68b9e057574b4c7a9879debf82e483423fdd4eed340e850a05696634a37d"}
Jul 13 21:35:18.727590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.727229    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f89b8757-lngxw" event={"ID":"29768d5e-b3f4-4820-a7ab-a9a2c621727a","Type":"ContainerDied","Data":"0a9fe6c328c83dd95cbd5ce26d924f2bd5342b1c710b9401a2c07463946225cb"}
Jul 13 21:35:18.727590 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.727250    2564 scope.go:117] "RemoveContainer" containerID="24cd68b9e057574b4c7a9879debf82e483423fdd4eed340e850a05696634a37d"
Jul 13 21:35:18.735160 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.735143    2564 scope.go:117] "RemoveContainer" containerID="24cd68b9e057574b4c7a9879debf82e483423fdd4eed340e850a05696634a37d"
Jul 13 21:35:18.735433 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:35:18.735414    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24cd68b9e057574b4c7a9879debf82e483423fdd4eed340e850a05696634a37d\": container with ID starting with 24cd68b9e057574b4c7a9879debf82e483423fdd4eed340e850a05696634a37d not found: ID does not exist" containerID="24cd68b9e057574b4c7a9879debf82e483423fdd4eed340e850a05696634a37d"
Jul 13 21:35:18.735477 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.735442    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24cd68b9e057574b4c7a9879debf82e483423fdd4eed340e850a05696634a37d"} err="failed to get container status \"24cd68b9e057574b4c7a9879debf82e483423fdd4eed340e850a05696634a37d\": rpc error: code = NotFound desc = could not find container \"24cd68b9e057574b4c7a9879debf82e483423fdd4eed340e850a05696634a37d\": container with ID starting with 24cd68b9e057574b4c7a9879debf82e483423fdd4eed340e850a05696634a37d not found: ID does not exist"
Jul 13 21:35:18.745189 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.745166    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67f89b8757-lngxw"]
Jul 13 21:35:18.746844 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:18.746824    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67f89b8757-lngxw"]
Jul 13 21:35:20.472154 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:20.472122    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29768d5e-b3f4-4820-a7ab-a9a2c621727a" path="/var/lib/kubelet/pods/29768d5e-b3f4-4820-a7ab-a9a2c621727a/volumes"
Jul 13 21:35:32.493491 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:32.493442    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:35:32.495782 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:32.495761    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b89f48b-c81d-40dc-a85d-ea77e98bc1f5-metrics-certs\") pod \"network-metrics-daemon-fjzfw\" (UID: \"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5\") " pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:35:32.670303 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:32.670270    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-89gzj\""
Jul 13 21:35:32.678250 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:32.678228    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fjzfw"
Jul 13 21:35:32.812379 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:32.812356    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fjzfw"]
Jul 13 21:35:32.815113 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:35:32.815083    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b89f48b_c81d_40dc_a85d_ea77e98bc1f5.slice/crio-6a346288ceebfdae0d37580d6c17dc30acfad4114fb4edcdec2b0db594943715 WatchSource:0}: Error finding container 6a346288ceebfdae0d37580d6c17dc30acfad4114fb4edcdec2b0db594943715: Status 404 returned error can't find the container with id 6a346288ceebfdae0d37580d6c17dc30acfad4114fb4edcdec2b0db594943715
Jul 13 21:35:33.787800 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:33.787765    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fjzfw" event={"ID":"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5","Type":"ContainerStarted","Data":"6a346288ceebfdae0d37580d6c17dc30acfad4114fb4edcdec2b0db594943715"}
Jul 13 21:35:34.793210 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:34.793177    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fjzfw" event={"ID":"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5","Type":"ContainerStarted","Data":"10bf0fc32fbcfe4d14a4b94417ee7a8c5a46ddf2357e573f76c96e83a823537c"}
Jul 13 21:35:34.793591 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:34.793231    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fjzfw" event={"ID":"4b89f48b-c81d-40dc-a85d-ea77e98bc1f5","Type":"ContainerStarted","Data":"b173ee6bf43c2e884f325da1ef22104b6e705e7bfebc6f42f85fd96b4aa89659"}
Jul 13 21:35:34.809111 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:34.809065    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fjzfw" podStartSLOduration=375.801751606 podStartE2EDuration="6m16.809048246s" totalImagesPullingTime="1.00729664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:35:34.808957864 +0000 UTC m=+376.731006782" watchObservedRunningTime="2026-07-13 21:35:34.809048246 +0000 UTC m=+376.731097165"
Jul 13 21:35:43.981881 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:43.981849    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:35:43.982353 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:43.982132    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="alertmanager" containerID="cri-o://3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9" gracePeriod=120
Jul 13 21:35:43.982353 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:43.982234    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="kube-rbac-proxy-metric" containerID="cri-o://9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f" gracePeriod=120
Jul 13 21:35:43.982353 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:43.982267    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="kube-rbac-proxy-web" containerID="cri-o://6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db" gracePeriod=120
Jul 13 21:35:43.982353 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:43.982294    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="config-reloader" containerID="cri-o://1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111" gracePeriod=120
Jul 13 21:35:43.982353 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:43.982306    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="prom-label-proxy" containerID="cri-o://8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6" gracePeriod=120
Jul 13 21:35:43.982600 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:43.982412    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="kube-rbac-proxy" containerID="cri-o://539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617" gracePeriod=120
Jul 13 21:35:44.839083 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:44.839045    2564 generic.go:356] "Generic (PLEG): container finished" podID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerID="8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6" exitCode=0
Jul 13 21:35:44.839083 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:44.839069    2564 generic.go:356] "Generic (PLEG): container finished" podID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerID="539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617" exitCode=0
Jul 13 21:35:44.839083 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:44.839076    2564 generic.go:356] "Generic (PLEG): container finished" podID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerID="1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111" exitCode=0
Jul 13 21:35:44.839083 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:44.839082    2564 generic.go:356] "Generic (PLEG): container finished" podID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerID="3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9" exitCode=0
Jul 13 21:35:44.839342 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:44.839116    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerDied","Data":"8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6"}
Jul 13 21:35:44.839342 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:44.839179    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerDied","Data":"539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617"}
Jul 13 21:35:44.839342 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:44.839205    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerDied","Data":"1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111"}
Jul 13 21:35:44.839342 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:44.839230    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerDied","Data":"3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9"}
Jul 13 21:35:45.253974 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.253950    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:45.295976 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.295947    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a18a26d9-75ab-4d36-b619-84b9e92a5721-alertmanager-main-db\") pod \"a18a26d9-75ab-4d36-b619-84b9e92a5721\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") "
Jul 13 21:35:45.295976 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.295982    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a18a26d9-75ab-4d36-b619-84b9e92a5721-tls-assets\") pod \"a18a26d9-75ab-4d36-b619-84b9e92a5721\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") "
Jul 13 21:35:45.296219 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.296008    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-cluster-tls-config\") pod \"a18a26d9-75ab-4d36-b619-84b9e92a5721\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") "
Jul 13 21:35:45.296219 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.296027    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4swsr\" (UniqueName: \"kubernetes.io/projected/a18a26d9-75ab-4d36-b619-84b9e92a5721-kube-api-access-4swsr\") pod \"a18a26d9-75ab-4d36-b619-84b9e92a5721\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") "
Jul 13 21:35:45.296219 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.296050    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a18a26d9-75ab-4d36-b619-84b9e92a5721-alertmanager-trusted-ca-bundle\") pod \"a18a26d9-75ab-4d36-b619-84b9e92a5721\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") "
Jul 13 21:35:45.296219 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.296081    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a18a26d9-75ab-4d36-b619-84b9e92a5721\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") "
Jul 13 21:35:45.296219 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.296112    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-main-tls\") pod \"a18a26d9-75ab-4d36-b619-84b9e92a5721\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") "
Jul 13 21:35:45.296219 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.296156    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-config-volume\") pod \"a18a26d9-75ab-4d36-b619-84b9e92a5721\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") "
Jul 13 21:35:45.296219 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.296177    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-web-config\") pod \"a18a26d9-75ab-4d36-b619-84b9e92a5721\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") "
Jul 13 21:35:45.296219 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.296200    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy-web\") pod \"a18a26d9-75ab-4d36-b619-84b9e92a5721\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") "
Jul 13 21:35:45.296680 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.296237    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a18a26d9-75ab-4d36-b619-84b9e92a5721-metrics-client-ca\") pod \"a18a26d9-75ab-4d36-b619-84b9e92a5721\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") "
Jul 13 21:35:45.296680 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.296285    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy\") pod \"a18a26d9-75ab-4d36-b619-84b9e92a5721\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") "
Jul 13 21:35:45.296680 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.296315    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a18a26d9-75ab-4d36-b619-84b9e92a5721-config-out\") pod \"a18a26d9-75ab-4d36-b619-84b9e92a5721\" (UID: \"a18a26d9-75ab-4d36-b619-84b9e92a5721\") "
Jul 13 21:35:45.296680 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.296323    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18a26d9-75ab-4d36-b619-84b9e92a5721-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a18a26d9-75ab-4d36-b619-84b9e92a5721" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:35:45.296680 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.296534    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a18a26d9-75ab-4d36-b619-84b9e92a5721-alertmanager-main-db\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:45.299342 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.299217    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a18a26d9-75ab-4d36-b619-84b9e92a5721" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:35:45.300683 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.300415    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a18a26d9-75ab-4d36-b619-84b9e92a5721-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a18a26d9-75ab-4d36-b619-84b9e92a5721" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:35:45.301148 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.300986    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18a26d9-75ab-4d36-b619-84b9e92a5721-config-out" (OuterVolumeSpecName: "config-out") pod "a18a26d9-75ab-4d36-b619-84b9e92a5721" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:35:45.301148 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.301086    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a18a26d9-75ab-4d36-b619-84b9e92a5721-kube-api-access-4swsr" (OuterVolumeSpecName: "kube-api-access-4swsr") pod "a18a26d9-75ab-4d36-b619-84b9e92a5721" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721"). InnerVolumeSpecName "kube-api-access-4swsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:35:45.301317 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.301279    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a18a26d9-75ab-4d36-b619-84b9e92a5721-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a18a26d9-75ab-4d36-b619-84b9e92a5721" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:35:45.302099 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.302072    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a18a26d9-75ab-4d36-b619-84b9e92a5721-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a18a26d9-75ab-4d36-b619-84b9e92a5721" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:35:45.302870 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.302838    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-config-volume" (OuterVolumeSpecName: "config-volume") pod "a18a26d9-75ab-4d36-b619-84b9e92a5721" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:35:45.302965 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.302863    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a18a26d9-75ab-4d36-b619-84b9e92a5721" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:35:45.303031 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.302981    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a18a26d9-75ab-4d36-b619-84b9e92a5721" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:35:45.303314 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.303279    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a18a26d9-75ab-4d36-b619-84b9e92a5721" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:35:45.305533 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.305512    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a18a26d9-75ab-4d36-b619-84b9e92a5721" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:35:45.314079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.314045    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-web-config" (OuterVolumeSpecName: "web-config") pod "a18a26d9-75ab-4d36-b619-84b9e92a5721" (UID: "a18a26d9-75ab-4d36-b619-84b9e92a5721"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:35:45.397622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.397543    2564 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-config-volume\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:45.397622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.397568    2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-web-config\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:45.397622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.397577    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:45.397622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.397587    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a18a26d9-75ab-4d36-b619-84b9e92a5721-metrics-client-ca\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:45.397622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.397597    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:45.397622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.397605    2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a18a26d9-75ab-4d36-b619-84b9e92a5721-config-out\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:45.397622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.397614    2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a18a26d9-75ab-4d36-b619-84b9e92a5721-tls-assets\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:45.397622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.397622    2564 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-cluster-tls-config\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:45.397622 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.397630    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4swsr\" (UniqueName: \"kubernetes.io/projected/a18a26d9-75ab-4d36-b619-84b9e92a5721-kube-api-access-4swsr\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:45.398014 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.397639    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a18a26d9-75ab-4d36-b619-84b9e92a5721-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:45.398014 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.397676    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:45.398014 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.397687    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a18a26d9-75ab-4d36-b619-84b9e92a5721-secret-alertmanager-main-tls\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:35:45.846222 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.846188    2564 generic.go:356] "Generic (PLEG): container finished" podID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerID="9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f" exitCode=0
Jul 13 21:35:45.846222 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.846212    2564 generic.go:356] "Generic (PLEG): container finished" podID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerID="6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db" exitCode=0
Jul 13 21:35:45.846453 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.846258    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerDied","Data":"9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f"}
Jul 13 21:35:45.846453 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.846268    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:45.846453 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.846316    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerDied","Data":"6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db"}
Jul 13 21:35:45.846453 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.846343    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a18a26d9-75ab-4d36-b619-84b9e92a5721","Type":"ContainerDied","Data":"9d4169b68075f35a170d8b418d348114b20fda69696c90d25016ee06ab9ddb70"}
Jul 13 21:35:45.846453 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.846373    2564 scope.go:117] "RemoveContainer" containerID="8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6"
Jul 13 21:35:45.854685 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.854666    2564 scope.go:117] "RemoveContainer" containerID="9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f"
Jul 13 21:35:45.862455 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.862433    2564 scope.go:117] "RemoveContainer" containerID="539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617"
Jul 13 21:35:45.869321 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.869296    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:35:45.870059 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.870037    2564 scope.go:117] "RemoveContainer" containerID="6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db"
Jul 13 21:35:45.873020 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.873000    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:35:45.877553 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.877535    2564 scope.go:117] "RemoveContainer" containerID="1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111"
Jul 13 21:35:45.884963 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.884946    2564 scope.go:117] "RemoveContainer" containerID="3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9"
Jul 13 21:35:45.892239 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.892220    2564 scope.go:117] "RemoveContainer" containerID="04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f"
Jul 13 21:35:45.897363 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897343    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:35:45.897686 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897671    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="kube-rbac-proxy-web"
Jul 13 21:35:45.897754 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897690    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="kube-rbac-proxy-web"
Jul 13 21:35:45.897754 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897701    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="prom-label-proxy"
Jul 13 21:35:45.897754 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897709    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="prom-label-proxy"
Jul 13 21:35:45.897754 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897724    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="alertmanager"
Jul 13 21:35:45.897754 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897733    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="alertmanager"
Jul 13 21:35:45.897754 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897745    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="kube-rbac-proxy-metric"
Jul 13 21:35:45.897754 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897753    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="kube-rbac-proxy-metric"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897765    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29768d5e-b3f4-4820-a7ab-a9a2c621727a" containerName="console"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897773    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="29768d5e-b3f4-4820-a7ab-a9a2c621727a" containerName="console"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897791    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="config-reloader"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897801    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="config-reloader"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897819    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="860f54f5-378e-478c-a414-dc546c02c05b" containerName="registry"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897826    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="860f54f5-378e-478c-a414-dc546c02c05b" containerName="registry"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897836    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="init-config-reloader"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897846    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="init-config-reloader"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897862    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="kube-rbac-proxy"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897870    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="kube-rbac-proxy"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897943    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="860f54f5-378e-478c-a414-dc546c02c05b" containerName="registry"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897955    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="kube-rbac-proxy-web"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897966    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="config-reloader"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897974    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="prom-label-proxy"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897983    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="kube-rbac-proxy-metric"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.897992    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="kube-rbac-proxy"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.898000    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="29768d5e-b3f4-4820-a7ab-a9a2c621727a" containerName="console"
Jul 13 21:35:45.898079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.898008    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" containerName="alertmanager"
Jul 13 21:35:45.900125 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.900108    2564 scope.go:117] "RemoveContainer" containerID="8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6"
Jul 13 21:35:45.900413 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:35:45.900394    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6\": container with ID starting with 8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6 not found: ID does not exist" containerID="8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6"
Jul 13 21:35:45.900488 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.900420    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6"} err="failed to get container status \"8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6\": rpc error: code = NotFound desc = could not find container \"8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6\": container with ID starting with 8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6 not found: ID does not exist"
Jul 13 21:35:45.900488 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.900438    2564 scope.go:117] "RemoveContainer" containerID="9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f"
Jul 13 21:35:45.902584 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:35:45.902557    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f\": container with ID starting with 9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f not found: ID does not exist" containerID="9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f"
Jul 13 21:35:45.902744 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.902593    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f"} err="failed to get container status \"9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f\": rpc error: code = NotFound desc = could not find container \"9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f\": container with ID starting with 9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f not found: ID does not exist"
Jul 13 21:35:45.902744 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.902614    2564 scope.go:117] "RemoveContainer" containerID="539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617"
Jul 13 21:35:45.902956 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:35:45.902936    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617\": container with ID starting with 539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617 not found: ID does not exist" containerID="539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617"
Jul 13 21:35:45.903060 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.902962    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617"} err="failed to get container status \"539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617\": rpc error: code = NotFound desc = could not find container \"539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617\": container with ID starting with 539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617 not found: ID does not exist"
Jul 13 21:35:45.903060 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.902978    2564 scope.go:117] "RemoveContainer" containerID="6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db"
Jul 13 21:35:45.903287 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:35:45.903261    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db\": container with ID starting with 6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db not found: ID does not exist" containerID="6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db"
Jul 13 21:35:45.903361 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.903296    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db"} err="failed to get container status \"6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db\": rpc error: code = NotFound desc = could not find container \"6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db\": container with ID starting with 6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db not found: ID does not exist"
Jul 13 21:35:45.903361 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.903316    2564 scope.go:117] "RemoveContainer" containerID="1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111"
Jul 13 21:35:45.903614 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:35:45.903585    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111\": container with ID starting with 1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111 not found: ID does not exist" containerID="1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111"
Jul 13 21:35:45.903730 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.903616    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:45.903730 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.903618    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111"} err="failed to get container status \"1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111\": rpc error: code = NotFound desc = could not find container \"1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111\": container with ID starting with 1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111 not found: ID does not exist"
Jul 13 21:35:45.903730 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.903640    2564 scope.go:117] "RemoveContainer" containerID="3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9"
Jul 13 21:35:45.904071 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:35:45.904051    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9\": container with ID starting with 3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9 not found: ID does not exist" containerID="3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9"
Jul 13 21:35:45.904148 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.904074    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9"} err="failed to get container status \"3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9\": rpc error: code = NotFound desc = could not find container \"3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9\": container with ID starting with 3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9 not found: ID does not exist"
Jul 13 21:35:45.904148 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.904092    2564 scope.go:117] "RemoveContainer" containerID="04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f"
Jul 13 21:35:45.904385 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:35:45.904357    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f\": container with ID starting with 04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f not found: ID does not exist" containerID="04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f"
Jul 13 21:35:45.906966 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.904700    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f"} err="failed to get container status \"04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f\": rpc error: code = NotFound desc = could not find container \"04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f\": container with ID starting with 04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f not found: ID does not exist"
Jul 13 21:35:45.906966 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.904755    2564 scope.go:117] "RemoveContainer" containerID="8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6"
Jul 13 21:35:45.906966 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.906120    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 13 21:35:45.906966 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.906424    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 13 21:35:45.906966 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.906448    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 13 21:35:45.906966 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.906637    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 13 21:35:45.906966 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.906790    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 13 21:35:45.906966 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.906873    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-r5bjd\""
Jul 13 21:35:45.906966 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.906880    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 13 21:35:45.910477 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.907609    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 13 21:35:45.910477 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.907640    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 13 21:35:45.910477 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.910144    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6"} err="failed to get container status \"8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6\": rpc error: code = NotFound desc = could not find container \"8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6\": container with ID starting with 8469380a1f831206ba65ebf1bec0ac942630420404c7e232b615af1282d431b6 not found: ID does not exist"
Jul 13 21:35:45.910477 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.910169    2564 scope.go:117] "RemoveContainer" containerID="9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f"
Jul 13 21:35:45.910737 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.910536    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f"} err="failed to get container status \"9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f\": rpc error: code = NotFound desc = could not find container \"9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f\": container with ID starting with 9f135a54ab9927273f29976fe40147f4518f71dc967018c6266349329b1e219f not found: ID does not exist"
Jul 13 21:35:45.910737 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.910566    2564 scope.go:117] "RemoveContainer" containerID="539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617"
Jul 13 21:35:45.911032 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.911004    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617"} err="failed to get container status \"539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617\": rpc error: code = NotFound desc = could not find container \"539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617\": container with ID starting with 539d7e7ec6948c15595503ca17eafbe48a96fb106a26c8b46664bb3300624617 not found: ID does not exist"
Jul 13 21:35:45.911122 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.911032    2564 scope.go:117] "RemoveContainer" containerID="6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db"
Jul 13 21:35:45.911782 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.911758    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db"} err="failed to get container status \"6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db\": rpc error: code = NotFound desc = could not find container \"6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db\": container with ID starting with 6fbfe2889f2f685163cea6af534686867cf9ef59fb64e6c450dd9a358c8399db not found: ID does not exist"
Jul 13 21:35:45.911782 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.911781    2564 scope.go:117] "RemoveContainer" containerID="1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111"
Jul 13 21:35:45.912078 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.912052    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111"} err="failed to get container status \"1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111\": rpc error: code = NotFound desc = could not find container \"1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111\": container with ID starting with 1f599c31c4131c0121c63df4132f3faaf6c88c96239517351122152d93450111 not found: ID does not exist"
Jul 13 21:35:45.912162 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.912079    2564 scope.go:117] "RemoveContainer" containerID="3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9"
Jul 13 21:35:45.912429 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.912409    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9"} err="failed to get container status \"3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9\": rpc error: code = NotFound desc = could not find container \"3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9\": container with ID starting with 3299330e73148f55b024ad62576c5cee1b50fdd8785855b1cfa7ff5e19a2f6a9 not found: ID does not exist"
Jul 13 21:35:45.912520 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.912431    2564 scope.go:117] "RemoveContainer" containerID="04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f"
Jul 13 21:35:45.912750 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.912728    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f"} err="failed to get container status \"04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f\": rpc error: code = NotFound desc = could not find container \"04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f\": container with ID starting with 04a838990266c8a1760839aa7c1b2e430cc1200f1011be6dad26536953593a4f not found: ID does not exist"
Jul 13 21:35:45.913130 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.913108    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 13 21:35:45.915917 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:45.915892    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:35:46.003940 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.003901    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-config-out\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.004138 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.003948    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-tls-assets\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.004138 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.003992    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.004138 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.004032    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.004138 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.004077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.004138 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.004106    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-config-volume\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.004138 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.004121    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2bd7\" (UniqueName: \"kubernetes.io/projected/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-kube-api-access-k2bd7\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.004384 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.004154    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.004384 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.004189    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.004384 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.004206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-web-config\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.004384 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.004219    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.004384 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.004246    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.004384 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.004267    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.104955 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.104857    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.104955 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.104910    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-config-out\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.104955 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.104938    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-tls-assets\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.104955 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.104955    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.105291 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.104974    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.105291 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.104992    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.105291 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.105020    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-config-volume\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.105291 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.105035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2bd7\" (UniqueName: \"kubernetes.io/projected/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-kube-api-access-k2bd7\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.105291 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.105054    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.105291 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.105097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.105795 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.105768    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.106050 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.106027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-web-config\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.106156 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.106064    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.106156 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.106098    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.108452 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.108019    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-config-out\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.108452 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.108075    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.108452 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.108126    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-tls-assets\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.108452 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.108185    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.108452 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.108326    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-config-volume\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.108452 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.108429    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.108855 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.108832    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.108932 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.108912    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.109016 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.108996    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.109061 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.109018    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.110336 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.110315    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-web-config\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.115233 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.115214    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2bd7\" (UniqueName: \"kubernetes.io/projected/adff7db5-7c76-44a4-b78a-14d7f4fbaa44-kube-api-access-k2bd7\") pod \"alertmanager-main-0\" (UID: \"adff7db5-7c76-44a4-b78a-14d7f4fbaa44\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.220054 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.220007    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:35:46.362243 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.362159    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:35:46.365659 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:35:46.365622    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadff7db5_7c76_44a4_b78a_14d7f4fbaa44.slice/crio-93534c05fc0821300f8743443e4215e9c2399b5db0dbae75bd07527a4fd14c55 WatchSource:0}: Error finding container 93534c05fc0821300f8743443e4215e9c2399b5db0dbae75bd07527a4fd14c55: Status 404 returned error can't find the container with id 93534c05fc0821300f8743443e4215e9c2399b5db0dbae75bd07527a4fd14c55
Jul 13 21:35:46.472210 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.472178    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a18a26d9-75ab-4d36-b619-84b9e92a5721" path="/var/lib/kubelet/pods/a18a26d9-75ab-4d36-b619-84b9e92a5721/volumes"
Jul 13 21:35:46.851633 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.851601    2564 generic.go:356] "Generic (PLEG): container finished" podID="adff7db5-7c76-44a4-b78a-14d7f4fbaa44" containerID="a7f01afc27a3f23cc9deb649d0f83efd2bbb09a0ae005323cc37599f693ba62d" exitCode=0
Jul 13 21:35:46.851796 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.851685    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"adff7db5-7c76-44a4-b78a-14d7f4fbaa44","Type":"ContainerDied","Data":"a7f01afc27a3f23cc9deb649d0f83efd2bbb09a0ae005323cc37599f693ba62d"}
Jul 13 21:35:46.851796 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:46.851760    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"adff7db5-7c76-44a4-b78a-14d7f4fbaa44","Type":"ContainerStarted","Data":"93534c05fc0821300f8743443e4215e9c2399b5db0dbae75bd07527a4fd14c55"}
Jul 13 21:35:47.125780 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.125745    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bfb4b4599-zrwlx"]
Jul 13 21:35:47.130432 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.130345    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.142136 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.142111    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bfb4b4599-zrwlx"]
Jul 13 21:35:47.215507 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.215478    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7z8k\" (UniqueName: \"kubernetes.io/projected/600934fd-3065-4065-b57f-48c6dafc69b2-kube-api-access-l7z8k\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.215638 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.215522    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-trusted-ca-bundle\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.215638 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.215544    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/600934fd-3065-4065-b57f-48c6dafc69b2-console-oauth-config\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.215638 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.215590    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-console-config\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.215638 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.215621    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/600934fd-3065-4065-b57f-48c6dafc69b2-console-serving-cert\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.215833 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.215642    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-service-ca\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.215833 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.215685    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-oauth-serving-cert\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.316177 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.316145    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/600934fd-3065-4065-b57f-48c6dafc69b2-console-oauth-config\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.316368 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.316196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-console-config\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.316368 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.316236    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/600934fd-3065-4065-b57f-48c6dafc69b2-console-serving-cert\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.316368 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.316265    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-service-ca\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.316368 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.316290    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-oauth-serving-cert\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.316368 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.316320    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7z8k\" (UniqueName: \"kubernetes.io/projected/600934fd-3065-4065-b57f-48c6dafc69b2-kube-api-access-l7z8k\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.316708 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.316355    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-trusted-ca-bundle\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.317168 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.317139    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-oauth-serving-cert\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.317282 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.317141    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-console-config\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.317282 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.317195    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-service-ca\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.317380 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.317361    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-trusted-ca-bundle\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.318753 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.318732    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/600934fd-3065-4065-b57f-48c6dafc69b2-console-serving-cert\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.318830 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.318809    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/600934fd-3065-4065-b57f-48c6dafc69b2-console-oauth-config\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.324137 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.324118    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7z8k\" (UniqueName: \"kubernetes.io/projected/600934fd-3065-4065-b57f-48c6dafc69b2-kube-api-access-l7z8k\") pod \"console-5bfb4b4599-zrwlx\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") " pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.456164 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.456068    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:47.585716 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.585679    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bfb4b4599-zrwlx"]
Jul 13 21:35:47.588035 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:35:47.588006    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600934fd_3065_4065_b57f_48c6dafc69b2.slice/crio-ebab14d0fb1b9f3068899cfe52e2c0d978c4cc13282cc77f3f0b8f008b14c488 WatchSource:0}: Error finding container ebab14d0fb1b9f3068899cfe52e2c0d978c4cc13282cc77f3f0b8f008b14c488: Status 404 returned error can't find the container with id ebab14d0fb1b9f3068899cfe52e2c0d978c4cc13282cc77f3f0b8f008b14c488
Jul 13 21:35:47.866064 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.866025    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"adff7db5-7c76-44a4-b78a-14d7f4fbaa44","Type":"ContainerStarted","Data":"b62206af2483943e4a1d01db470f329373f6d4509ff448d315d709a0ea3df7db"}
Jul 13 21:35:47.866269 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.866141    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"adff7db5-7c76-44a4-b78a-14d7f4fbaa44","Type":"ContainerStarted","Data":"bd77ac27c4ab9f9487b95a35dd13f35b89d11642caff6379b8c30136f588eb1d"}
Jul 13 21:35:47.866269 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.866174    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"adff7db5-7c76-44a4-b78a-14d7f4fbaa44","Type":"ContainerStarted","Data":"36a7f562f534704bcd230e2d581f9d7752978be90c54b14794d527faef04a025"}
Jul 13 21:35:47.866269 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.866199    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"adff7db5-7c76-44a4-b78a-14d7f4fbaa44","Type":"ContainerStarted","Data":"9be7b708abb51064df1564f6b95218e8d2c7212bb583aeb8c3dde11119332e82"}
Jul 13 21:35:47.866269 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.866232    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"adff7db5-7c76-44a4-b78a-14d7f4fbaa44","Type":"ContainerStarted","Data":"5a9d638b82fb635455f495ed483162e86b5de9fe97b52bdf3f0fca1d174cc7a3"}
Jul 13 21:35:47.866269 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.866263    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"adff7db5-7c76-44a4-b78a-14d7f4fbaa44","Type":"ContainerStarted","Data":"b36c23508058f7cb7a74ead94677618c8b9398dd4d3ec9f00f4bfe6bc1ec256f"}
Jul 13 21:35:47.867617 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.867595    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bfb4b4599-zrwlx" event={"ID":"600934fd-3065-4065-b57f-48c6dafc69b2","Type":"ContainerStarted","Data":"59bbd996a3d08267b7ef22f3ddaf4384c5e6a1250e0f0b0b5c3c67ce2417ba7e"}
Jul 13 21:35:47.867731 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.867632    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bfb4b4599-zrwlx" event={"ID":"600934fd-3065-4065-b57f-48c6dafc69b2","Type":"ContainerStarted","Data":"ebab14d0fb1b9f3068899cfe52e2c0d978c4cc13282cc77f3f0b8f008b14c488"}
Jul 13 21:35:47.892257 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.892206    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.877666289 podStartE2EDuration="2.892188204s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.521915ms" isStatefulPod=false podCreationTimestamp="2026-07-13 21:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:35:47.889784987 +0000 UTC m=+389.811833905" watchObservedRunningTime="2026-07-13 21:35:47.892188204 +0000 UTC m=+389.814237136"
Jul 13 21:35:47.907852 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:47.907791    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bfb4b4599-zrwlx" podStartSLOduration=0.907769705 podStartE2EDuration="907.769705ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:35:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:35:47.906352085 +0000 UTC m=+389.828401004" watchObservedRunningTime="2026-07-13 21:35:47.907769705 +0000 UTC m=+389.829818623"
Jul 13 21:35:57.456426 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:57.456380    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:57.456910 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:57.456483    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:57.461187 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:57.461164    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:57.918778 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:57.918749    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:35:57.962806 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:35:57.962775    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-987dd7958-z7kbl"]
Jul 13 21:36:22.983965 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:22.983903    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-987dd7958-z7kbl" podUID="7ff9f1ba-d7c1-4012-a917-88d586ff29ac" containerName="console" containerID="cri-o://ae7558fc07237c63aeaeb82606c7919d10424576429f28eee0fa43984ed69ac9" gracePeriod=15
Jul 13 21:36:23.232445 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.232422    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:36:23.322387 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.322353    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v25jd\" (UniqueName: \"kubernetes.io/projected/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-kube-api-access-v25jd\") pod \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") "
Jul 13 21:36:23.322387 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.322391    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-trusted-ca-bundle\") pod \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") "
Jul 13 21:36:23.322665 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.322436    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-config\") pod \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") "
Jul 13 21:36:23.322665 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.322457    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-serving-cert\") pod \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") "
Jul 13 21:36:23.322665 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.322484    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-service-ca\") pod \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") "
Jul 13 21:36:23.322665 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.322511    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-oauth-config\") pod \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") "
Jul 13 21:36:23.322665 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.322553    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-oauth-serving-cert\") pod \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\" (UID: \"7ff9f1ba-d7c1-4012-a917-88d586ff29ac\") "
Jul 13 21:36:23.322933 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.322882    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-config" (OuterVolumeSpecName: "console-config") pod "7ff9f1ba-d7c1-4012-a917-88d586ff29ac" (UID: "7ff9f1ba-d7c1-4012-a917-88d586ff29ac"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:36:23.323006 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.322983    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7ff9f1ba-d7c1-4012-a917-88d586ff29ac" (UID: "7ff9f1ba-d7c1-4012-a917-88d586ff29ac"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:36:23.323067 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.323046    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-service-ca" (OuterVolumeSpecName: "service-ca") pod "7ff9f1ba-d7c1-4012-a917-88d586ff29ac" (UID: "7ff9f1ba-d7c1-4012-a917-88d586ff29ac"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:36:23.323141 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.323124    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7ff9f1ba-d7c1-4012-a917-88d586ff29ac" (UID: "7ff9f1ba-d7c1-4012-a917-88d586ff29ac"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:36:23.324623 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.324592    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-kube-api-access-v25jd" (OuterVolumeSpecName: "kube-api-access-v25jd") pod "7ff9f1ba-d7c1-4012-a917-88d586ff29ac" (UID: "7ff9f1ba-d7c1-4012-a917-88d586ff29ac"). InnerVolumeSpecName "kube-api-access-v25jd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:36:23.324623 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.324592    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7ff9f1ba-d7c1-4012-a917-88d586ff29ac" (UID: "7ff9f1ba-d7c1-4012-a917-88d586ff29ac"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:36:23.324815 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.324627    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7ff9f1ba-d7c1-4012-a917-88d586ff29ac" (UID: "7ff9f1ba-d7c1-4012-a917-88d586ff29ac"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:36:23.423527 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.423489    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v25jd\" (UniqueName: \"kubernetes.io/projected/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-kube-api-access-v25jd\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:36:23.423527 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.423519    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-trusted-ca-bundle\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:36:23.423527 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.423528    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-config\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:36:23.423527 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.423537    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-serving-cert\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:36:23.423843 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.423547    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-service-ca\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:36:23.423843 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.423555    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-console-oauth-config\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:36:23.423843 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:23.423564    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ff9f1ba-d7c1-4012-a917-88d586ff29ac-oauth-serving-cert\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:36:24.025429 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:24.025392    2564 generic.go:356] "Generic (PLEG): container finished" podID="7ff9f1ba-d7c1-4012-a917-88d586ff29ac" containerID="ae7558fc07237c63aeaeb82606c7919d10424576429f28eee0fa43984ed69ac9" exitCode=0
Jul 13 21:36:24.025872 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:24.025447    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-987dd7958-z7kbl"
Jul 13 21:36:24.025872 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:24.025475    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-987dd7958-z7kbl" event={"ID":"7ff9f1ba-d7c1-4012-a917-88d586ff29ac","Type":"ContainerDied","Data":"ae7558fc07237c63aeaeb82606c7919d10424576429f28eee0fa43984ed69ac9"}
Jul 13 21:36:24.025872 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:24.025530    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-987dd7958-z7kbl" event={"ID":"7ff9f1ba-d7c1-4012-a917-88d586ff29ac","Type":"ContainerDied","Data":"2114e9e2c8de8179de76c0c2a25c4176c8f41f0924c6877122345a7095b9bb3f"}
Jul 13 21:36:24.025872 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:24.025552    2564 scope.go:117] "RemoveContainer" containerID="ae7558fc07237c63aeaeb82606c7919d10424576429f28eee0fa43984ed69ac9"
Jul 13 21:36:24.034631 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:24.034609    2564 scope.go:117] "RemoveContainer" containerID="ae7558fc07237c63aeaeb82606c7919d10424576429f28eee0fa43984ed69ac9"
Jul 13 21:36:24.034913 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:36:24.034896    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae7558fc07237c63aeaeb82606c7919d10424576429f28eee0fa43984ed69ac9\": container with ID starting with ae7558fc07237c63aeaeb82606c7919d10424576429f28eee0fa43984ed69ac9 not found: ID does not exist" containerID="ae7558fc07237c63aeaeb82606c7919d10424576429f28eee0fa43984ed69ac9"
Jul 13 21:36:24.034970 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:24.034922    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae7558fc07237c63aeaeb82606c7919d10424576429f28eee0fa43984ed69ac9"} err="failed to get container status \"ae7558fc07237c63aeaeb82606c7919d10424576429f28eee0fa43984ed69ac9\": rpc error: code = NotFound desc = could not find container \"ae7558fc07237c63aeaeb82606c7919d10424576429f28eee0fa43984ed69ac9\": container with ID starting with ae7558fc07237c63aeaeb82606c7919d10424576429f28eee0fa43984ed69ac9 not found: ID does not exist"
Jul 13 21:36:24.046164 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:24.046136    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-987dd7958-z7kbl"]
Jul 13 21:36:24.049491 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:24.049469    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-987dd7958-z7kbl"]
Jul 13 21:36:24.472465 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:36:24.472384    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ff9f1ba-d7c1-4012-a917-88d586ff29ac" path="/var/lib/kubelet/pods/7ff9f1ba-d7c1-4012-a917-88d586ff29ac/volumes"
Jul 13 21:37:30.027578 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.027541    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-txgfs"]
Jul 13 21:37:30.028032 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.027899    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ff9f1ba-d7c1-4012-a917-88d586ff29ac" containerName="console"
Jul 13 21:37:30.028032 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.027912    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ff9f1ba-d7c1-4012-a917-88d586ff29ac" containerName="console"
Jul 13 21:37:30.028032 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.027980    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ff9f1ba-d7c1-4012-a917-88d586ff29ac" containerName="console"
Jul 13 21:37:30.031910 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.031890    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-txgfs"
Jul 13 21:37:30.034313 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.034292    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 13 21:37:30.034313 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.034304    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 13 21:37:30.035185 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.035172    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-pwkhq\""
Jul 13 21:37:30.037562 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.037543    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-txgfs"]
Jul 13 21:37:30.163274 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.163236    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51c7295e-8dbb-466b-9843-8084a4c2be76-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-txgfs\" (UID: \"51c7295e-8dbb-466b-9843-8084a4c2be76\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-txgfs"
Jul 13 21:37:30.163463 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.163323    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vj57\" (UniqueName: \"kubernetes.io/projected/51c7295e-8dbb-466b-9843-8084a4c2be76-kube-api-access-7vj57\") pod \"cert-manager-cainjector-559cdd8c87-txgfs\" (UID: \"51c7295e-8dbb-466b-9843-8084a4c2be76\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-txgfs"
Jul 13 21:37:30.264198 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.264163    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51c7295e-8dbb-466b-9843-8084a4c2be76-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-txgfs\" (UID: \"51c7295e-8dbb-466b-9843-8084a4c2be76\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-txgfs"
Jul 13 21:37:30.264385 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.264223    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vj57\" (UniqueName: \"kubernetes.io/projected/51c7295e-8dbb-466b-9843-8084a4c2be76-kube-api-access-7vj57\") pod \"cert-manager-cainjector-559cdd8c87-txgfs\" (UID: \"51c7295e-8dbb-466b-9843-8084a4c2be76\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-txgfs"
Jul 13 21:37:30.272346 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.272315    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vj57\" (UniqueName: \"kubernetes.io/projected/51c7295e-8dbb-466b-9843-8084a4c2be76-kube-api-access-7vj57\") pod \"cert-manager-cainjector-559cdd8c87-txgfs\" (UID: \"51c7295e-8dbb-466b-9843-8084a4c2be76\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-txgfs"
Jul 13 21:37:30.272718 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.272697    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51c7295e-8dbb-466b-9843-8084a4c2be76-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-txgfs\" (UID: \"51c7295e-8dbb-466b-9843-8084a4c2be76\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-txgfs"
Jul 13 21:37:30.356581 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.356463    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-txgfs"
Jul 13 21:37:30.494747 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:30.494722    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-txgfs"]
Jul 13 21:37:30.497261 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:37:30.497235    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51c7295e_8dbb_466b_9843_8084a4c2be76.slice/crio-0758fb77cb043a51e2e0bc7e429006eac4f82f72776f44381b6b00af8861fb12 WatchSource:0}: Error finding container 0758fb77cb043a51e2e0bc7e429006eac4f82f72776f44381b6b00af8861fb12: Status 404 returned error can't find the container with id 0758fb77cb043a51e2e0bc7e429006eac4f82f72776f44381b6b00af8861fb12
Jul 13 21:37:31.298759 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:31.298715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-txgfs" event={"ID":"51c7295e-8dbb-466b-9843-8084a4c2be76","Type":"ContainerStarted","Data":"0758fb77cb043a51e2e0bc7e429006eac4f82f72776f44381b6b00af8861fb12"}
Jul 13 21:37:33.720325 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:33.720214    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lt52x"]
Jul 13 21:37:33.723789 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:33.723762    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lt52x"
Jul 13 21:37:33.726120 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:33.726095    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-dhfbc\""
Jul 13 21:37:33.732255 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:33.732230    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lt52x"]
Jul 13 21:37:33.898170 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:33.898134    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73214c6a-4df8-4d56-a01c-2aeb7e3f2052-bound-sa-token\") pod \"cert-manager-55b4d954d5-lt52x\" (UID: \"73214c6a-4df8-4d56-a01c-2aeb7e3f2052\") " pod="cert-manager/cert-manager-55b4d954d5-lt52x"
Jul 13 21:37:33.898360 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:33.898202    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwfqf\" (UniqueName: \"kubernetes.io/projected/73214c6a-4df8-4d56-a01c-2aeb7e3f2052-kube-api-access-dwfqf\") pod \"cert-manager-55b4d954d5-lt52x\" (UID: \"73214c6a-4df8-4d56-a01c-2aeb7e3f2052\") " pod="cert-manager/cert-manager-55b4d954d5-lt52x"
Jul 13 21:37:33.999413 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:33.999330    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73214c6a-4df8-4d56-a01c-2aeb7e3f2052-bound-sa-token\") pod \"cert-manager-55b4d954d5-lt52x\" (UID: \"73214c6a-4df8-4d56-a01c-2aeb7e3f2052\") " pod="cert-manager/cert-manager-55b4d954d5-lt52x"
Jul 13 21:37:33.999413 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:33.999388    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwfqf\" (UniqueName: \"kubernetes.io/projected/73214c6a-4df8-4d56-a01c-2aeb7e3f2052-kube-api-access-dwfqf\") pod \"cert-manager-55b4d954d5-lt52x\" (UID: \"73214c6a-4df8-4d56-a01c-2aeb7e3f2052\") " pod="cert-manager/cert-manager-55b4d954d5-lt52x"
Jul 13 21:37:34.007468 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:34.007425    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73214c6a-4df8-4d56-a01c-2aeb7e3f2052-bound-sa-token\") pod \"cert-manager-55b4d954d5-lt52x\" (UID: \"73214c6a-4df8-4d56-a01c-2aeb7e3f2052\") " pod="cert-manager/cert-manager-55b4d954d5-lt52x"
Jul 13 21:37:34.007625 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:34.007499    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwfqf\" (UniqueName: \"kubernetes.io/projected/73214c6a-4df8-4d56-a01c-2aeb7e3f2052-kube-api-access-dwfqf\") pod \"cert-manager-55b4d954d5-lt52x\" (UID: \"73214c6a-4df8-4d56-a01c-2aeb7e3f2052\") " pod="cert-manager/cert-manager-55b4d954d5-lt52x"
Jul 13 21:37:34.034554 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:34.034501    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lt52x"
Jul 13 21:37:34.196712 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:34.196677    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lt52x"]
Jul 13 21:37:34.200484 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:37:34.200449    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73214c6a_4df8_4d56_a01c_2aeb7e3f2052.slice/crio-89f9178e2db4508922259497df0a317fc61dca4e5d0a0bdd6e66437e753e73dc WatchSource:0}: Error finding container 89f9178e2db4508922259497df0a317fc61dca4e5d0a0bdd6e66437e753e73dc: Status 404 returned error can't find the container with id 89f9178e2db4508922259497df0a317fc61dca4e5d0a0bdd6e66437e753e73dc
Jul 13 21:37:34.315250 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:34.315199    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lt52x" event={"ID":"73214c6a-4df8-4d56-a01c-2aeb7e3f2052","Type":"ContainerStarted","Data":"f2d21f132041e55643641f82ebce5264fa33ce222aa8bf4bfd903ddf93d6676b"}
Jul 13 21:37:34.315250 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:34.315257    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lt52x" event={"ID":"73214c6a-4df8-4d56-a01c-2aeb7e3f2052","Type":"ContainerStarted","Data":"89f9178e2db4508922259497df0a317fc61dca4e5d0a0bdd6e66437e753e73dc"}
Jul 13 21:37:34.316780 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:34.316752    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-txgfs" event={"ID":"51c7295e-8dbb-466b-9843-8084a4c2be76","Type":"ContainerStarted","Data":"f4ae4f2a990e6d8939c067a3d277041216481ac27a91b6836bf834a19cb26ca4"}
Jul 13 21:37:34.330164 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:34.330101    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-lt52x" podStartSLOduration=1.330079032 podStartE2EDuration="1.330079032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:37:34.329619789 +0000 UTC m=+496.251668717" watchObservedRunningTime="2026-07-13 21:37:34.330079032 +0000 UTC m=+496.252127950"
Jul 13 21:37:34.345594 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:34.345541    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-txgfs" podStartSLOduration=1.422123024 podStartE2EDuration="4.345521953s" totalImagesPullingTime="2.923398929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:37:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:37:34.343781842 +0000 UTC m=+496.265830771" watchObservedRunningTime="2026-07-13 21:37:34.345521953 +0000 UTC m=+496.267570871"
Jul 13 21:37:35.229377 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:35.229340    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w9gh8"]
Jul 13 21:37:35.232821 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:35.232796    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-w9gh8"
Jul 13 21:37:35.235049 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:35.235030    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-8q4lp\""
Jul 13 21:37:35.239775 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:35.239752    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w9gh8"]
Jul 13 21:37:35.412891 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:35.412847    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4efd3a4a-9adc-41d4-a171-1275cb4c424f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w9gh8\" (UID: \"4efd3a4a-9adc-41d4-a171-1275cb4c424f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w9gh8"
Jul 13 21:37:35.412891 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:35.412896    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qscd8\" (UniqueName: \"kubernetes.io/projected/4efd3a4a-9adc-41d4-a171-1275cb4c424f-kube-api-access-qscd8\") pod \"cert-manager-webhook-6486ffd997-w9gh8\" (UID: \"4efd3a4a-9adc-41d4-a171-1275cb4c424f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w9gh8"
Jul 13 21:37:35.513869 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:35.513832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4efd3a4a-9adc-41d4-a171-1275cb4c424f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w9gh8\" (UID: \"4efd3a4a-9adc-41d4-a171-1275cb4c424f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w9gh8"
Jul 13 21:37:35.514046 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:35.513877    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qscd8\" (UniqueName: \"kubernetes.io/projected/4efd3a4a-9adc-41d4-a171-1275cb4c424f-kube-api-access-qscd8\") pod \"cert-manager-webhook-6486ffd997-w9gh8\" (UID: \"4efd3a4a-9adc-41d4-a171-1275cb4c424f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w9gh8"
Jul 13 21:37:35.524162 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:35.524135    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4efd3a4a-9adc-41d4-a171-1275cb4c424f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w9gh8\" (UID: \"4efd3a4a-9adc-41d4-a171-1275cb4c424f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w9gh8"
Jul 13 21:37:35.524301 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:35.524282    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qscd8\" (UniqueName: \"kubernetes.io/projected/4efd3a4a-9adc-41d4-a171-1275cb4c424f-kube-api-access-qscd8\") pod \"cert-manager-webhook-6486ffd997-w9gh8\" (UID: \"4efd3a4a-9adc-41d4-a171-1275cb4c424f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w9gh8"
Jul 13 21:37:35.543192 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:35.543158    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-w9gh8"
Jul 13 21:37:35.681139 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:35.681011    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w9gh8"]
Jul 13 21:37:35.683776 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:37:35.683751    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4efd3a4a_9adc_41d4_a171_1275cb4c424f.slice/crio-3b07dc6bae33c9bd878e0cc186251bbbe0c1a04c43eafb443838e58d52576b26 WatchSource:0}: Error finding container 3b07dc6bae33c9bd878e0cc186251bbbe0c1a04c43eafb443838e58d52576b26: Status 404 returned error can't find the container with id 3b07dc6bae33c9bd878e0cc186251bbbe0c1a04c43eafb443838e58d52576b26
Jul 13 21:37:36.325950 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:36.325910    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-w9gh8" event={"ID":"4efd3a4a-9adc-41d4-a171-1275cb4c424f","Type":"ContainerStarted","Data":"93abaf55e7f63f9f4ec4f3c78c763fe2323e758f73cabdb3766819427b9cb907"}
Jul 13 21:37:36.325950 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:36.325955    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-w9gh8" event={"ID":"4efd3a4a-9adc-41d4-a171-1275cb4c424f","Type":"ContainerStarted","Data":"3b07dc6bae33c9bd878e0cc186251bbbe0c1a04c43eafb443838e58d52576b26"}
Jul 13 21:37:36.326438 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:36.326040    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-w9gh8"
Jul 13 21:37:36.341289 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:36.341235    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-w9gh8" podStartSLOduration=1.341218297 podStartE2EDuration="1.341218297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:37:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:37:36.339316302 +0000 UTC m=+498.261365231" watchObservedRunningTime="2026-07-13 21:37:36.341218297 +0000 UTC m=+498.263267214"
Jul 13 21:37:42.332416 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:42.332374    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-w9gh8"
Jul 13 21:37:58.904956 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:58.904913    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"]
Jul 13 21:37:58.910022 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:58.909988    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:58.913637 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:58.913606    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 13 21:37:58.913637 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:58.913618    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 13 21:37:58.913877 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:58.913860    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 13 21:37:58.913939 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:58.913866    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 13 21:37:58.914215 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:58.914192    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6rfkx\""
Jul 13 21:37:58.943017 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:58.942986    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"]
Jul 13 21:37:59.007771 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.007729    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-manifests\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.007964 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.007831    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-tmp\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.007964 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.007894    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-webhook-cert\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.007964 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.007923    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzc76\" (UniqueName: \"kubernetes.io/projected/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-kube-api-access-mzc76\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.008095 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.007965    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.109138 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.109100    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-manifests\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.109382 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.109151    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-tmp\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.109382 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.109180    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-webhook-cert\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.109509 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.109389    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzc76\" (UniqueName: \"kubernetes.io/projected/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-kube-api-access-mzc76\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.109509 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.109426    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.109612 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.109576    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-manifests\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.111445 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.111413    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-tmp\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.111708 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.111689    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-webhook-cert\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.111782 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.111696    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.117709 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.117690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzc76\" (UniqueName: \"kubernetes.io/projected/b5916b8f-caaf-4c3e-91e2-4bfe6599a603-kube-api-access-mzc76\") pod \"opendatahub-operator-controller-manager-5db8865497-6jvg6\" (UID: \"b5916b8f-caaf-4c3e-91e2-4bfe6599a603\") " pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.220861 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.220758    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:37:59.366499 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.366475    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"]
Jul 13 21:37:59.368967 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:37:59.368933    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5916b8f_caaf_4c3e_91e2_4bfe6599a603.slice/crio-029b2feaaaa5c36f5addb7cbc7646dd4fce36c84e3aa4dd70aa9dcbe453def32 WatchSource:0}: Error finding container 029b2feaaaa5c36f5addb7cbc7646dd4fce36c84e3aa4dd70aa9dcbe453def32: Status 404 returned error can't find the container with id 029b2feaaaa5c36f5addb7cbc7646dd4fce36c84e3aa4dd70aa9dcbe453def32
Jul 13 21:37:59.431245 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:37:59.431208    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6" event={"ID":"b5916b8f-caaf-4c3e-91e2-4bfe6599a603","Type":"ContainerStarted","Data":"029b2feaaaa5c36f5addb7cbc7646dd4fce36c84e3aa4dd70aa9dcbe453def32"}
Jul 13 21:38:03.454699 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:03.454662    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6" event={"ID":"b5916b8f-caaf-4c3e-91e2-4bfe6599a603","Type":"ContainerStarted","Data":"73c83c7d6a8ce87481d27caa12193b0bb16df365fcc23fc54de3d07910c0523c"}
Jul 13 21:38:04.461528 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:04.461487    2564 generic.go:356] "Generic (PLEG): container finished" podID="b5916b8f-caaf-4c3e-91e2-4bfe6599a603" containerID="73c83c7d6a8ce87481d27caa12193b0bb16df365fcc23fc54de3d07910c0523c" exitCode=0
Jul 13 21:38:04.462008 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:04.461555    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6" event={"ID":"b5916b8f-caaf-4c3e-91e2-4bfe6599a603","Type":"ContainerDied","Data":"73c83c7d6a8ce87481d27caa12193b0bb16df365fcc23fc54de3d07910c0523c"}
Jul 13 21:38:05.467807 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:05.467775    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6" event={"ID":"b5916b8f-caaf-4c3e-91e2-4bfe6599a603","Type":"ContainerStarted","Data":"bf8f088fec3627847fad388dce2a41ccec8fb9fd43ad352cef5b72c0761c567f"}
Jul 13 21:38:05.468193 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:05.467855    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:38:05.489549 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:05.489499    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6" podStartSLOduration=2.862420447 podStartE2EDuration="7.489482784s" totalImagesPullingTime="3.559540335s" totalInitContainerRuntime="1.067522002s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:37:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:38:05.487294845 +0000 UTC m=+527.409343763" watchObservedRunningTime="2026-07-13 21:38:05.489482784 +0000 UTC m=+527.411531701"
Jul 13 21:38:16.476744 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:16.476675    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5db8865497-6jvg6"
Jul 13 21:38:18.893554 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:18.893517    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"]
Jul 13 21:38:18.898357 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:18.898340    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"
Jul 13 21:38:18.902797 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:18.902774    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tk8zb\""
Jul 13 21:38:18.903430 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:18.903404    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 13 21:38:18.903572 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:18.903557    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 13 21:38:18.903850 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:18.903832    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 13 21:38:18.912039 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:18.912019    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 13 21:38:18.928875 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:18.928848    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"]
Jul 13 21:38:18.980739 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:18.980708    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a21ba74e-705e-47cf-a195-d9f3c75bd808-tmp\") pod \"kube-auth-proxy-6bff766844-g2vz4\" (UID: \"a21ba74e-705e-47cf-a195-d9f3c75bd808\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"
Jul 13 21:38:18.980894 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:18.980746    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a21ba74e-705e-47cf-a195-d9f3c75bd808-tls-certs\") pod \"kube-auth-proxy-6bff766844-g2vz4\" (UID: \"a21ba74e-705e-47cf-a195-d9f3c75bd808\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"
Jul 13 21:38:18.980894 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:18.980769    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z87vg\" (UniqueName: \"kubernetes.io/projected/a21ba74e-705e-47cf-a195-d9f3c75bd808-kube-api-access-z87vg\") pod \"kube-auth-proxy-6bff766844-g2vz4\" (UID: \"a21ba74e-705e-47cf-a195-d9f3c75bd808\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"
Jul 13 21:38:19.081800 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:19.081760    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a21ba74e-705e-47cf-a195-d9f3c75bd808-tmp\") pod \"kube-auth-proxy-6bff766844-g2vz4\" (UID: \"a21ba74e-705e-47cf-a195-d9f3c75bd808\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"
Jul 13 21:38:19.081996 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:19.081812    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a21ba74e-705e-47cf-a195-d9f3c75bd808-tls-certs\") pod \"kube-auth-proxy-6bff766844-g2vz4\" (UID: \"a21ba74e-705e-47cf-a195-d9f3c75bd808\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"
Jul 13 21:38:19.081996 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:19.081843    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z87vg\" (UniqueName: \"kubernetes.io/projected/a21ba74e-705e-47cf-a195-d9f3c75bd808-kube-api-access-z87vg\") pod \"kube-auth-proxy-6bff766844-g2vz4\" (UID: \"a21ba74e-705e-47cf-a195-d9f3c75bd808\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"
Jul 13 21:38:19.084116 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:19.084086    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a21ba74e-705e-47cf-a195-d9f3c75bd808-tmp\") pod \"kube-auth-proxy-6bff766844-g2vz4\" (UID: \"a21ba74e-705e-47cf-a195-d9f3c75bd808\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"
Jul 13 21:38:19.084292 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:19.084274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a21ba74e-705e-47cf-a195-d9f3c75bd808-tls-certs\") pod \"kube-auth-proxy-6bff766844-g2vz4\" (UID: \"a21ba74e-705e-47cf-a195-d9f3c75bd808\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"
Jul 13 21:38:19.092074 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:19.092053    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z87vg\" (UniqueName: \"kubernetes.io/projected/a21ba74e-705e-47cf-a195-d9f3c75bd808-kube-api-access-z87vg\") pod \"kube-auth-proxy-6bff766844-g2vz4\" (UID: \"a21ba74e-705e-47cf-a195-d9f3c75bd808\") " pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"
Jul 13 21:38:19.211384 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:19.211300    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"
Jul 13 21:38:19.356683 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:19.356623    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bff766844-g2vz4"]
Jul 13 21:38:19.358851 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:38:19.358827    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda21ba74e_705e_47cf_a195_d9f3c75bd808.slice/crio-71158592ec3188594dadda063a9018eb73f45b4eb20ea591974136a73c4015a4 WatchSource:0}: Error finding container 71158592ec3188594dadda063a9018eb73f45b4eb20ea591974136a73c4015a4: Status 404 returned error can't find the container with id 71158592ec3188594dadda063a9018eb73f45b4eb20ea591974136a73c4015a4
Jul 13 21:38:19.530199 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:19.530164    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4" event={"ID":"a21ba74e-705e-47cf-a195-d9f3c75bd808","Type":"ContainerStarted","Data":"71158592ec3188594dadda063a9018eb73f45b4eb20ea591974136a73c4015a4"}
Jul 13 21:38:22.546439 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:22.546402    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4" event={"ID":"a21ba74e-705e-47cf-a195-d9f3c75bd808","Type":"ContainerStarted","Data":"3582387756ba8dfc57efdf6208480d90d6436de011e1dc81298944eb9155a0b7"}
Jul 13 21:38:22.563268 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:22.563209    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bff766844-g2vz4" podStartSLOduration=1.521910867 podStartE2EDuration="4.563186764s" totalImagesPullingTime="3.041275897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:38:22.562194105 +0000 UTC m=+544.484243022" watchObservedRunningTime="2026-07-13 21:38:22.563186764 +0000 UTC m=+544.485235682"
Jul 13 21:38:26.104454 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.104420    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"]
Jul 13 21:38:26.107951 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.107928    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.110228 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.110206    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 13 21:38:26.110935 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.110915    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 13 21:38:26.111133 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.111104    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7htvz\""
Jul 13 21:38:26.111258 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.111162    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:38:26.111258 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.111172    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 13 21:38:26.111258 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.111224    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 13 21:38:26.118357 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.118332    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"]
Jul 13 21:38:26.242028 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.241990    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f03eee92-cf1d-4b4e-9a89-60ce8ef9733f-manager-config\") pod \"lws-controller-manager-789595b4db-srkbn\" (UID: \"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f\") " pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.242028 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.242030    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f03eee92-cf1d-4b4e-9a89-60ce8ef9733f-metrics-cert\") pod \"lws-controller-manager-789595b4db-srkbn\" (UID: \"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f\") " pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.242294 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.242076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jbvs\" (UniqueName: \"kubernetes.io/projected/f03eee92-cf1d-4b4e-9a89-60ce8ef9733f-kube-api-access-6jbvs\") pod \"lws-controller-manager-789595b4db-srkbn\" (UID: \"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f\") " pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.242294 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.242165    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f03eee92-cf1d-4b4e-9a89-60ce8ef9733f-cert\") pod \"lws-controller-manager-789595b4db-srkbn\" (UID: \"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f\") " pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.343635 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.343596    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f03eee92-cf1d-4b4e-9a89-60ce8ef9733f-manager-config\") pod \"lws-controller-manager-789595b4db-srkbn\" (UID: \"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f\") " pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.343844 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.343669    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f03eee92-cf1d-4b4e-9a89-60ce8ef9733f-metrics-cert\") pod \"lws-controller-manager-789595b4db-srkbn\" (UID: \"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f\") " pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.343844 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.343735    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jbvs\" (UniqueName: \"kubernetes.io/projected/f03eee92-cf1d-4b4e-9a89-60ce8ef9733f-kube-api-access-6jbvs\") pod \"lws-controller-manager-789595b4db-srkbn\" (UID: \"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f\") " pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.343844 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.343786    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f03eee92-cf1d-4b4e-9a89-60ce8ef9733f-cert\") pod \"lws-controller-manager-789595b4db-srkbn\" (UID: \"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f\") " pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.344498 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.344459    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f03eee92-cf1d-4b4e-9a89-60ce8ef9733f-manager-config\") pod \"lws-controller-manager-789595b4db-srkbn\" (UID: \"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f\") " pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.346262 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.346244    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f03eee92-cf1d-4b4e-9a89-60ce8ef9733f-cert\") pod \"lws-controller-manager-789595b4db-srkbn\" (UID: \"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f\") " pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.346359 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.346310    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f03eee92-cf1d-4b4e-9a89-60ce8ef9733f-metrics-cert\") pod \"lws-controller-manager-789595b4db-srkbn\" (UID: \"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f\") " pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.358085 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.358022    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jbvs\" (UniqueName: \"kubernetes.io/projected/f03eee92-cf1d-4b4e-9a89-60ce8ef9733f-kube-api-access-6jbvs\") pod \"lws-controller-manager-789595b4db-srkbn\" (UID: \"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f\") " pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.418403 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.418359    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:26.554901 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.554857    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"]
Jul 13 21:38:26.557604 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:38:26.557575    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf03eee92_cf1d_4b4e_9a89_60ce8ef9733f.slice/crio-52ab94ed38ae8cb55a7e6b98c2310c9fd5b168e35b76c44ae8fc04e6be65abf8 WatchSource:0}: Error finding container 52ab94ed38ae8cb55a7e6b98c2310c9fd5b168e35b76c44ae8fc04e6be65abf8: Status 404 returned error can't find the container with id 52ab94ed38ae8cb55a7e6b98c2310c9fd5b168e35b76c44ae8fc04e6be65abf8
Jul 13 21:38:26.566172 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:26.566147    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn" event={"ID":"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f","Type":"ContainerStarted","Data":"52ab94ed38ae8cb55a7e6b98c2310c9fd5b168e35b76c44ae8fc04e6be65abf8"}
Jul 13 21:38:29.585252 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:29.585219    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn" event={"ID":"f03eee92-cf1d-4b4e-9a89-60ce8ef9733f","Type":"ContainerStarted","Data":"56c4da4c899fa816fe3f1edf0b83ae07238f447fbed2714a817443bbd1c29544"}
Jul 13 21:38:29.585672 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:29.585325    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:38:29.628243 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:29.628187    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn" podStartSLOduration=1.016699189 podStartE2EDuration="3.62816702s" totalImagesPullingTime="2.611467831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:38:29.624232049 +0000 UTC m=+551.546280967" watchObservedRunningTime="2026-07-13 21:38:29.62816702 +0000 UTC m=+551.550215936"
Jul 13 21:38:40.592957 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:38:40.592928    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-789595b4db-srkbn"
Jul 13 21:39:18.406382 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:18.406353    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:39:18.408111 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:18.408081    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:39:43.226989 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.226900    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cf88bc554-vb7gt"]
Jul 13 21:39:43.230482 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.230457    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.240405 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.240377    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cf88bc554-vb7gt"]
Jul 13 21:39:43.299161 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.299118    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00194f71-93a9-452e-8224-f1275f8e19a8-console-config\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.299357 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.299177    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00194f71-93a9-452e-8224-f1275f8e19a8-oauth-serving-cert\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.299357 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.299214    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00194f71-93a9-452e-8224-f1275f8e19a8-console-serving-cert\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.299357 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.299230    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00194f71-93a9-452e-8224-f1275f8e19a8-service-ca\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.299357 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.299254    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00194f71-93a9-452e-8224-f1275f8e19a8-trusted-ca-bundle\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.299357 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.299293    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgv9t\" (UniqueName: \"kubernetes.io/projected/00194f71-93a9-452e-8224-f1275f8e19a8-kube-api-access-lgv9t\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.299357 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.299319    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00194f71-93a9-452e-8224-f1275f8e19a8-console-oauth-config\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.400238 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.400195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00194f71-93a9-452e-8224-f1275f8e19a8-oauth-serving-cert\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.400238 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.400238    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00194f71-93a9-452e-8224-f1275f8e19a8-console-serving-cert\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.400511 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.400256    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00194f71-93a9-452e-8224-f1275f8e19a8-service-ca\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.400511 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.400284    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00194f71-93a9-452e-8224-f1275f8e19a8-trusted-ca-bundle\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.400511 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.400302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgv9t\" (UniqueName: \"kubernetes.io/projected/00194f71-93a9-452e-8224-f1275f8e19a8-kube-api-access-lgv9t\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.400511 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.400332    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00194f71-93a9-452e-8224-f1275f8e19a8-console-oauth-config\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.400511 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.400364    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00194f71-93a9-452e-8224-f1275f8e19a8-console-config\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.401055 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.401025    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00194f71-93a9-452e-8224-f1275f8e19a8-oauth-serving-cert\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.401175 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.401103    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00194f71-93a9-452e-8224-f1275f8e19a8-console-config\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.401175 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.401113    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00194f71-93a9-452e-8224-f1275f8e19a8-service-ca\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.401311 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.401286    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00194f71-93a9-452e-8224-f1275f8e19a8-trusted-ca-bundle\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.402933 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.402915    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00194f71-93a9-452e-8224-f1275f8e19a8-console-oauth-config\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.402989 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.402963    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00194f71-93a9-452e-8224-f1275f8e19a8-console-serving-cert\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.408152 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.408129    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgv9t\" (UniqueName: \"kubernetes.io/projected/00194f71-93a9-452e-8224-f1275f8e19a8-kube-api-access-lgv9t\") pod \"console-6cf88bc554-vb7gt\" (UID: \"00194f71-93a9-452e-8224-f1275f8e19a8\") " pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.540738 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.540697    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:43.679461 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.679437    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cf88bc554-vb7gt"]
Jul 13 21:39:43.933756 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.933644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cf88bc554-vb7gt" event={"ID":"00194f71-93a9-452e-8224-f1275f8e19a8","Type":"ContainerStarted","Data":"96312602ce1ab05ab8062f06c8ae396dcbb3a71fff1ab5932f29538d8c1a49b7"}
Jul 13 21:39:43.933756 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.933705    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cf88bc554-vb7gt" event={"ID":"00194f71-93a9-452e-8224-f1275f8e19a8","Type":"ContainerStarted","Data":"c9354787eab60767c8450bb411b779da69e751b6625eb9ca9fba5c353a32a088"}
Jul 13 21:39:43.952106 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:43.952063    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cf88bc554-vb7gt" podStartSLOduration=0.952047399 podStartE2EDuration="952.047399ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:39:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:39:43.950638142 +0000 UTC m=+625.872687062" watchObservedRunningTime="2026-07-13 21:39:43.952047399 +0000 UTC m=+625.874096317"
Jul 13 21:39:53.541344 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:53.541305    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:53.541848 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:53.541359    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:53.546170 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:53.546146    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:53.981402 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:53.981310    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cf88bc554-vb7gt"
Jul 13 21:39:54.028008 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:39:54.027975    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bfb4b4599-zrwlx"]
Jul 13 21:40:19.049904 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.049861    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5bfb4b4599-zrwlx" podUID="600934fd-3065-4065-b57f-48c6dafc69b2" containerName="console" containerID="cri-o://59bbd996a3d08267b7ef22f3ddaf4384c5e6a1250e0f0b0b5c3c67ce2417ba7e" gracePeriod=15
Jul 13 21:40:19.314071 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.314008    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:40:19.408378 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.408340    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-trusted-ca-bundle\") pod \"600934fd-3065-4065-b57f-48c6dafc69b2\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") "
Jul 13 21:40:19.408575 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.408387    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7z8k\" (UniqueName: \"kubernetes.io/projected/600934fd-3065-4065-b57f-48c6dafc69b2-kube-api-access-l7z8k\") pod \"600934fd-3065-4065-b57f-48c6dafc69b2\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") "
Jul 13 21:40:19.408575 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.408410    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/600934fd-3065-4065-b57f-48c6dafc69b2-console-serving-cert\") pod \"600934fd-3065-4065-b57f-48c6dafc69b2\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") "
Jul 13 21:40:19.408575 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.408434    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/600934fd-3065-4065-b57f-48c6dafc69b2-console-oauth-config\") pod \"600934fd-3065-4065-b57f-48c6dafc69b2\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") "
Jul 13 21:40:19.408575 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.408464    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-service-ca\") pod \"600934fd-3065-4065-b57f-48c6dafc69b2\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") "
Jul 13 21:40:19.408575 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.408490    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-oauth-serving-cert\") pod \"600934fd-3065-4065-b57f-48c6dafc69b2\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") "
Jul 13 21:40:19.408575 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.408571    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-console-config\") pod \"600934fd-3065-4065-b57f-48c6dafc69b2\" (UID: \"600934fd-3065-4065-b57f-48c6dafc69b2\") "
Jul 13 21:40:19.408968 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.408913    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "600934fd-3065-4065-b57f-48c6dafc69b2" (UID: "600934fd-3065-4065-b57f-48c6dafc69b2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:40:19.409087 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.409002    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-service-ca" (OuterVolumeSpecName: "service-ca") pod "600934fd-3065-4065-b57f-48c6dafc69b2" (UID: "600934fd-3065-4065-b57f-48c6dafc69b2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:40:19.409261 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.409066    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "600934fd-3065-4065-b57f-48c6dafc69b2" (UID: "600934fd-3065-4065-b57f-48c6dafc69b2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:40:19.409261 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.409242    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-console-config" (OuterVolumeSpecName: "console-config") pod "600934fd-3065-4065-b57f-48c6dafc69b2" (UID: "600934fd-3065-4065-b57f-48c6dafc69b2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:40:19.411013 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.410978    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/600934fd-3065-4065-b57f-48c6dafc69b2-kube-api-access-l7z8k" (OuterVolumeSpecName: "kube-api-access-l7z8k") pod "600934fd-3065-4065-b57f-48c6dafc69b2" (UID: "600934fd-3065-4065-b57f-48c6dafc69b2"). InnerVolumeSpecName "kube-api-access-l7z8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:40:19.411200 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.411169    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/600934fd-3065-4065-b57f-48c6dafc69b2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "600934fd-3065-4065-b57f-48c6dafc69b2" (UID: "600934fd-3065-4065-b57f-48c6dafc69b2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:40:19.411285 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.411250    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/600934fd-3065-4065-b57f-48c6dafc69b2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "600934fd-3065-4065-b57f-48c6dafc69b2" (UID: "600934fd-3065-4065-b57f-48c6dafc69b2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:40:19.509900 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.509861    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-trusted-ca-bundle\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:40:19.509900 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.509891    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7z8k\" (UniqueName: \"kubernetes.io/projected/600934fd-3065-4065-b57f-48c6dafc69b2-kube-api-access-l7z8k\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:40:19.509900 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.509901    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/600934fd-3065-4065-b57f-48c6dafc69b2-console-serving-cert\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:40:19.509900 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.509910    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/600934fd-3065-4065-b57f-48c6dafc69b2-console-oauth-config\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:40:19.510162 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.509920    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-service-ca\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:40:19.510162 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.509928    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-oauth-serving-cert\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:40:19.510162 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:19.509936    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/600934fd-3065-4065-b57f-48c6dafc69b2-console-config\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:40:20.110331 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:20.110292    2564 generic.go:356] "Generic (PLEG): container finished" podID="600934fd-3065-4065-b57f-48c6dafc69b2" containerID="59bbd996a3d08267b7ef22f3ddaf4384c5e6a1250e0f0b0b5c3c67ce2417ba7e" exitCode=0
Jul 13 21:40:20.110849 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:20.110340    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bfb4b4599-zrwlx"
Jul 13 21:40:20.110849 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:20.110363    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bfb4b4599-zrwlx" event={"ID":"600934fd-3065-4065-b57f-48c6dafc69b2","Type":"ContainerDied","Data":"59bbd996a3d08267b7ef22f3ddaf4384c5e6a1250e0f0b0b5c3c67ce2417ba7e"}
Jul 13 21:40:20.110849 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:20.110402    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bfb4b4599-zrwlx" event={"ID":"600934fd-3065-4065-b57f-48c6dafc69b2","Type":"ContainerDied","Data":"ebab14d0fb1b9f3068899cfe52e2c0d978c4cc13282cc77f3f0b8f008b14c488"}
Jul 13 21:40:20.110849 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:20.110421    2564 scope.go:117] "RemoveContainer" containerID="59bbd996a3d08267b7ef22f3ddaf4384c5e6a1250e0f0b0b5c3c67ce2417ba7e"
Jul 13 21:40:20.119830 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:20.119813    2564 scope.go:117] "RemoveContainer" containerID="59bbd996a3d08267b7ef22f3ddaf4384c5e6a1250e0f0b0b5c3c67ce2417ba7e"
Jul 13 21:40:20.120123 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:40:20.120105    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59bbd996a3d08267b7ef22f3ddaf4384c5e6a1250e0f0b0b5c3c67ce2417ba7e\": container with ID starting with 59bbd996a3d08267b7ef22f3ddaf4384c5e6a1250e0f0b0b5c3c67ce2417ba7e not found: ID does not exist" containerID="59bbd996a3d08267b7ef22f3ddaf4384c5e6a1250e0f0b0b5c3c67ce2417ba7e"
Jul 13 21:40:20.120165 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:20.120134    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59bbd996a3d08267b7ef22f3ddaf4384c5e6a1250e0f0b0b5c3c67ce2417ba7e"} err="failed to get container status \"59bbd996a3d08267b7ef22f3ddaf4384c5e6a1250e0f0b0b5c3c67ce2417ba7e\": rpc error: code = NotFound desc = could not find container \"59bbd996a3d08267b7ef22f3ddaf4384c5e6a1250e0f0b0b5c3c67ce2417ba7e\": container with ID starting with 59bbd996a3d08267b7ef22f3ddaf4384c5e6a1250e0f0b0b5c3c67ce2417ba7e not found: ID does not exist"
Jul 13 21:40:20.132259 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:20.132227    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bfb4b4599-zrwlx"]
Jul 13 21:40:20.134433 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:20.134409    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bfb4b4599-zrwlx"]
Jul 13 21:40:20.473016 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:20.472936    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="600934fd-3065-4065-b57f-48c6dafc69b2" path="/var/lib/kubelet/pods/600934fd-3065-4065-b57f-48c6dafc69b2/volumes"
Jul 13 21:40:28.679621 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.679582    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"]
Jul 13 21:40:28.680110 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.679948    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600934fd-3065-4065-b57f-48c6dafc69b2" containerName="console"
Jul 13 21:40:28.680110 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.679959    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="600934fd-3065-4065-b57f-48c6dafc69b2" containerName="console"
Jul 13 21:40:28.680110 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.680038    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="600934fd-3065-4065-b57f-48c6dafc69b2" containerName="console"
Jul 13 21:40:28.683069 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.683051    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"
Jul 13 21:40:28.685934 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.685907    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 21:40:28.686081 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.686043    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 21:40:28.686387 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.686373    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v6jfl\""
Jul 13 21:40:28.699723 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.699701    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"]
Jul 13 21:40:28.791226 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.791195    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4ncz\" (UniqueName: \"kubernetes.io/projected/3d83dd88-8cb6-4280-9e7c-996aec39dd8e-kube-api-access-g4ncz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vqnw9\" (UID: \"3d83dd88-8cb6-4280-9e7c-996aec39dd8e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"
Jul 13 21:40:28.791401 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.791255    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d83dd88-8cb6-4280-9e7c-996aec39dd8e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vqnw9\" (UID: \"3d83dd88-8cb6-4280-9e7c-996aec39dd8e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"
Jul 13 21:40:28.892624 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.892587    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4ncz\" (UniqueName: \"kubernetes.io/projected/3d83dd88-8cb6-4280-9e7c-996aec39dd8e-kube-api-access-g4ncz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vqnw9\" (UID: \"3d83dd88-8cb6-4280-9e7c-996aec39dd8e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"
Jul 13 21:40:28.892815 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.892735    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d83dd88-8cb6-4280-9e7c-996aec39dd8e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vqnw9\" (UID: \"3d83dd88-8cb6-4280-9e7c-996aec39dd8e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"
Jul 13 21:40:28.893207 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.893186    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d83dd88-8cb6-4280-9e7c-996aec39dd8e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vqnw9\" (UID: \"3d83dd88-8cb6-4280-9e7c-996aec39dd8e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"
Jul 13 21:40:28.902789 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.902762    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4ncz\" (UniqueName: \"kubernetes.io/projected/3d83dd88-8cb6-4280-9e7c-996aec39dd8e-kube-api-access-g4ncz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vqnw9\" (UID: \"3d83dd88-8cb6-4280-9e7c-996aec39dd8e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"
Jul 13 21:40:28.993670 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:28.993569    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"
Jul 13 21:40:29.132595 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:29.132561    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"]
Jul 13 21:40:29.136733 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:40:29.136704    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d83dd88_8cb6_4280_9e7c_996aec39dd8e.slice/crio-783b74161f5f75e1d3a11ae0e39f027d66467000c7f88c7b3c52e6570dfd6f7e WatchSource:0}: Error finding container 783b74161f5f75e1d3a11ae0e39f027d66467000c7f88c7b3c52e6570dfd6f7e: Status 404 returned error can't find the container with id 783b74161f5f75e1d3a11ae0e39f027d66467000c7f88c7b3c52e6570dfd6f7e
Jul 13 21:40:29.139312 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:29.139294    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:40:29.153610 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:29.153575    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9" event={"ID":"3d83dd88-8cb6-4280-9e7c-996aec39dd8e","Type":"ContainerStarted","Data":"783b74161f5f75e1d3a11ae0e39f027d66467000c7f88c7b3c52e6570dfd6f7e"}
Jul 13 21:40:29.615845 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:29.615811    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"]
Jul 13 21:40:29.620889 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:29.620855    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"]
Jul 13 21:40:34.181706 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:34.181662    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9" podUID="3d83dd88-8cb6-4280-9e7c-996aec39dd8e" containerName="manager" containerID="cri-o://8e185c69aa199d8e9ce8047807732a3d24f36fd177990bb019b75411157e19f0" gracePeriod=2
Jul 13 21:40:34.433532 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:34.433508    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"
Jul 13 21:40:34.435633 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:40:34.435603    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vqnw9\" is forbidden: User \"system:node:ip-10-0-140-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-98.ec2.internal' and this object" podUID="3d83dd88-8cb6-4280-9e7c-996aec39dd8e" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"
Jul 13 21:40:34.544277 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:34.544234    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4ncz\" (UniqueName: \"kubernetes.io/projected/3d83dd88-8cb6-4280-9e7c-996aec39dd8e-kube-api-access-g4ncz\") pod \"3d83dd88-8cb6-4280-9e7c-996aec39dd8e\" (UID: \"3d83dd88-8cb6-4280-9e7c-996aec39dd8e\") "
Jul 13 21:40:34.544451 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:34.544315    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d83dd88-8cb6-4280-9e7c-996aec39dd8e-extensions-socket-volume\") pod \"3d83dd88-8cb6-4280-9e7c-996aec39dd8e\" (UID: \"3d83dd88-8cb6-4280-9e7c-996aec39dd8e\") "
Jul 13 21:40:34.544593 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:34.544567    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d83dd88-8cb6-4280-9e7c-996aec39dd8e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3d83dd88-8cb6-4280-9e7c-996aec39dd8e" (UID: "3d83dd88-8cb6-4280-9e7c-996aec39dd8e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:40:34.546542 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:34.546520    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d83dd88-8cb6-4280-9e7c-996aec39dd8e-kube-api-access-g4ncz" (OuterVolumeSpecName: "kube-api-access-g4ncz") pod "3d83dd88-8cb6-4280-9e7c-996aec39dd8e" (UID: "3d83dd88-8cb6-4280-9e7c-996aec39dd8e"). InnerVolumeSpecName "kube-api-access-g4ncz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:40:34.645138 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:34.645099    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d83dd88-8cb6-4280-9e7c-996aec39dd8e-extensions-socket-volume\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:40:34.645138 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:34.645132    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4ncz\" (UniqueName: \"kubernetes.io/projected/3d83dd88-8cb6-4280-9e7c-996aec39dd8e-kube-api-access-g4ncz\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:40:35.187877 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:35.187838    2564 generic.go:356] "Generic (PLEG): container finished" podID="3d83dd88-8cb6-4280-9e7c-996aec39dd8e" containerID="8e185c69aa199d8e9ce8047807732a3d24f36fd177990bb019b75411157e19f0" exitCode=2
Jul 13 21:40:35.187877 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:35.187878    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"
Jul 13 21:40:35.188333 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:35.187928    2564 scope.go:117] "RemoveContainer" containerID="8e185c69aa199d8e9ce8047807732a3d24f36fd177990bb019b75411157e19f0"
Jul 13 21:40:35.197290 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:35.197272    2564 scope.go:117] "RemoveContainer" containerID="8e185c69aa199d8e9ce8047807732a3d24f36fd177990bb019b75411157e19f0"
Jul 13 21:40:35.197573 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:40:35.197548    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e185c69aa199d8e9ce8047807732a3d24f36fd177990bb019b75411157e19f0\": container with ID starting with 8e185c69aa199d8e9ce8047807732a3d24f36fd177990bb019b75411157e19f0 not found: ID does not exist" containerID="8e185c69aa199d8e9ce8047807732a3d24f36fd177990bb019b75411157e19f0"
Jul 13 21:40:35.197696 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:35.197582    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e185c69aa199d8e9ce8047807732a3d24f36fd177990bb019b75411157e19f0"} err="failed to get container status \"8e185c69aa199d8e9ce8047807732a3d24f36fd177990bb019b75411157e19f0\": rpc error: code = NotFound desc = could not find container \"8e185c69aa199d8e9ce8047807732a3d24f36fd177990bb019b75411157e19f0\": container with ID starting with 8e185c69aa199d8e9ce8047807732a3d24f36fd177990bb019b75411157e19f0 not found: ID does not exist"
Jul 13 21:40:35.197696 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:40:35.197671    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vqnw9\" is forbidden: User \"system:node:ip-10-0-140-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-98.ec2.internal' and this object" podUID="3d83dd88-8cb6-4280-9e7c-996aec39dd8e" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vqnw9"
Jul 13 21:40:36.473055 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:36.473012    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d83dd88-8cb6-4280-9e7c-996aec39dd8e" path="/var/lib/kubelet/pods/3d83dd88-8cb6-4280-9e7c-996aec39dd8e/volumes"
Jul 13 21:40:46.044550 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.044512    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"]
Jul 13 21:40:46.044933 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.044866    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d83dd88-8cb6-4280-9e7c-996aec39dd8e" containerName="manager"
Jul 13 21:40:46.044933 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.044877    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d83dd88-8cb6-4280-9e7c-996aec39dd8e" containerName="manager"
Jul 13 21:40:46.045012 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.044936    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d83dd88-8cb6-4280-9e7c-996aec39dd8e" containerName="manager"
Jul 13 21:40:46.048073 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.048054    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"
Jul 13 21:40:46.050840 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.050811    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 21:40:46.051077 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.051059    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v6jfl\""
Jul 13 21:40:46.051142 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.051084    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 21:40:46.060160 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.060136    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"]
Jul 13 21:40:46.149358 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.149316    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79bpd\" (UniqueName: \"kubernetes.io/projected/48474f36-7987-43b5-a9ac-e2461b54258e-kube-api-access-79bpd\") pod \"kuadrant-operator-controller-manager-86878c769f-2fvtn\" (UID: \"48474f36-7987-43b5-a9ac-e2461b54258e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"
Jul 13 21:40:46.149548 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.149372    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48474f36-7987-43b5-a9ac-e2461b54258e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2fvtn\" (UID: \"48474f36-7987-43b5-a9ac-e2461b54258e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"
Jul 13 21:40:46.250609 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.250575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79bpd\" (UniqueName: \"kubernetes.io/projected/48474f36-7987-43b5-a9ac-e2461b54258e-kube-api-access-79bpd\") pod \"kuadrant-operator-controller-manager-86878c769f-2fvtn\" (UID: \"48474f36-7987-43b5-a9ac-e2461b54258e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"
Jul 13 21:40:46.250801 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.250618    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48474f36-7987-43b5-a9ac-e2461b54258e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2fvtn\" (UID: \"48474f36-7987-43b5-a9ac-e2461b54258e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"
Jul 13 21:40:46.250998 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.250979    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48474f36-7987-43b5-a9ac-e2461b54258e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2fvtn\" (UID: \"48474f36-7987-43b5-a9ac-e2461b54258e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"
Jul 13 21:40:46.258769 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.258741    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79bpd\" (UniqueName: \"kubernetes.io/projected/48474f36-7987-43b5-a9ac-e2461b54258e-kube-api-access-79bpd\") pod \"kuadrant-operator-controller-manager-86878c769f-2fvtn\" (UID: \"48474f36-7987-43b5-a9ac-e2461b54258e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"
Jul 13 21:40:46.359027 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.358926    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"
Jul 13 21:40:46.503623 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:46.503597    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"]
Jul 13 21:40:46.506433 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:40:46.506401    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48474f36_7987_43b5_a9ac_e2461b54258e.slice/crio-3f01e39015fd95f468b45e040c15a77464b540b0c53e767de7095544927c979b WatchSource:0}: Error finding container 3f01e39015fd95f468b45e040c15a77464b540b0c53e767de7095544927c979b: Status 404 returned error can't find the container with id 3f01e39015fd95f468b45e040c15a77464b540b0c53e767de7095544927c979b
Jul 13 21:40:47.242959 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:47.242926    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn" event={"ID":"48474f36-7987-43b5-a9ac-e2461b54258e","Type":"ContainerStarted","Data":"3cdeb2b7de2992fd766ea82d428562a8ae74850c0f9c37bf67788a52155fe3ff"}
Jul 13 21:40:47.243399 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:47.242978    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn" event={"ID":"48474f36-7987-43b5-a9ac-e2461b54258e","Type":"ContainerStarted","Data":"3f01e39015fd95f468b45e040c15a77464b540b0c53e767de7095544927c979b"}
Jul 13 21:40:47.243399 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:47.243026    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"
Jul 13 21:40:47.280727 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:47.280670    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn" podStartSLOduration=1.280641951 podStartE2EDuration="1.280641951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:40:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:40:47.279007342 +0000 UTC m=+689.201056258" watchObservedRunningTime="2026-07-13 21:40:47.280641951 +0000 UTC m=+689.202690870"
Jul 13 21:40:58.250293 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:40:58.250263    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"
Jul 13 21:41:19.665356 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:19.665324    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-4c7zl"]
Jul 13 21:41:19.675471 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:19.675437    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4c7zl"
Jul 13 21:41:19.676845 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:19.676819    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4c7zl"]
Jul 13 21:41:19.677792 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:19.677773    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cpbz4\""
Jul 13 21:41:19.856845 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:19.856804    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srzfg\" (UniqueName: \"kubernetes.io/projected/b58e70c0-96b4-43d2-887c-f3c4e748fa02-kube-api-access-srzfg\") pod \"authorino-58cd955675-4c7zl\" (UID: \"b58e70c0-96b4-43d2-887c-f3c4e748fa02\") " pod="kuadrant-system/authorino-58cd955675-4c7zl"
Jul 13 21:41:19.958115 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:19.958014    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srzfg\" (UniqueName: \"kubernetes.io/projected/b58e70c0-96b4-43d2-887c-f3c4e748fa02-kube-api-access-srzfg\") pod \"authorino-58cd955675-4c7zl\" (UID: \"b58e70c0-96b4-43d2-887c-f3c4e748fa02\") " pod="kuadrant-system/authorino-58cd955675-4c7zl"
Jul 13 21:41:19.969255 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:19.969221    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srzfg\" (UniqueName: \"kubernetes.io/projected/b58e70c0-96b4-43d2-887c-f3c4e748fa02-kube-api-access-srzfg\") pod \"authorino-58cd955675-4c7zl\" (UID: \"b58e70c0-96b4-43d2-887c-f3c4e748fa02\") " pod="kuadrant-system/authorino-58cd955675-4c7zl"
Jul 13 21:41:19.986632 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:19.986600    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4c7zl"
Jul 13 21:41:20.121291 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:20.121265    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4c7zl"]
Jul 13 21:41:20.123435 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:41:20.123402    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb58e70c0_96b4_43d2_887c_f3c4e748fa02.slice/crio-925f589b2986cf84f4bb7f2298f5f037263a79da1342e6357c574127395e6d80 WatchSource:0}: Error finding container 925f589b2986cf84f4bb7f2298f5f037263a79da1342e6357c574127395e6d80: Status 404 returned error can't find the container with id 925f589b2986cf84f4bb7f2298f5f037263a79da1342e6357c574127395e6d80
Jul 13 21:41:20.399033 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:20.398996    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4c7zl" event={"ID":"b58e70c0-96b4-43d2-887c-f3c4e748fa02","Type":"ContainerStarted","Data":"925f589b2986cf84f4bb7f2298f5f037263a79da1342e6357c574127395e6d80"}
Jul 13 21:41:24.422276 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:24.422242    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4c7zl" event={"ID":"b58e70c0-96b4-43d2-887c-f3c4e748fa02","Type":"ContainerStarted","Data":"04ee9653019317176111eda5e4c491054e8cbb07d43f56fa7b42a3ce048b914d"}
Jul 13 21:41:24.434987 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:24.434933    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-4c7zl" podStartSLOduration=2.035125219 podStartE2EDuration="5.434913984s" totalImagesPullingTime="3.399788765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:41:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:41:24.434422332 +0000 UTC m=+726.356471246" watchObservedRunningTime="2026-07-13 21:41:24.434913984 +0000 UTC m=+726.356962902"
Jul 13 21:41:25.255090 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:25.255057    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4c7zl"]
Jul 13 21:41:26.433373 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:26.433338    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-4c7zl" podUID="b58e70c0-96b4-43d2-887c-f3c4e748fa02" containerName="authorino" containerID="cri-o://04ee9653019317176111eda5e4c491054e8cbb07d43f56fa7b42a3ce048b914d" gracePeriod=30
Jul 13 21:41:26.690525 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:26.690448    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4c7zl"
Jul 13 21:41:26.705904 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:26.705872    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srzfg\" (UniqueName: \"kubernetes.io/projected/b58e70c0-96b4-43d2-887c-f3c4e748fa02-kube-api-access-srzfg\") pod \"b58e70c0-96b4-43d2-887c-f3c4e748fa02\" (UID: \"b58e70c0-96b4-43d2-887c-f3c4e748fa02\") "
Jul 13 21:41:26.708134 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:26.708106    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b58e70c0-96b4-43d2-887c-f3c4e748fa02-kube-api-access-srzfg" (OuterVolumeSpecName: "kube-api-access-srzfg") pod "b58e70c0-96b4-43d2-887c-f3c4e748fa02" (UID: "b58e70c0-96b4-43d2-887c-f3c4e748fa02"). InnerVolumeSpecName "kube-api-access-srzfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:41:26.806511 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:26.806475    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srzfg\" (UniqueName: \"kubernetes.io/projected/b58e70c0-96b4-43d2-887c-f3c4e748fa02-kube-api-access-srzfg\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:41:27.438580 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:27.438546    2564 generic.go:356] "Generic (PLEG): container finished" podID="b58e70c0-96b4-43d2-887c-f3c4e748fa02" containerID="04ee9653019317176111eda5e4c491054e8cbb07d43f56fa7b42a3ce048b914d" exitCode=0
Jul 13 21:41:27.439034 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:27.438591    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4c7zl"
Jul 13 21:41:27.439034 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:27.438617    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4c7zl" event={"ID":"b58e70c0-96b4-43d2-887c-f3c4e748fa02","Type":"ContainerDied","Data":"04ee9653019317176111eda5e4c491054e8cbb07d43f56fa7b42a3ce048b914d"}
Jul 13 21:41:27.439034 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:27.438673    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4c7zl" event={"ID":"b58e70c0-96b4-43d2-887c-f3c4e748fa02","Type":"ContainerDied","Data":"925f589b2986cf84f4bb7f2298f5f037263a79da1342e6357c574127395e6d80"}
Jul 13 21:41:27.439034 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:27.438691    2564 scope.go:117] "RemoveContainer" containerID="04ee9653019317176111eda5e4c491054e8cbb07d43f56fa7b42a3ce048b914d"
Jul 13 21:41:27.447788 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:27.447768    2564 scope.go:117] "RemoveContainer" containerID="04ee9653019317176111eda5e4c491054e8cbb07d43f56fa7b42a3ce048b914d"
Jul 13 21:41:27.448070 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:41:27.448051    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ee9653019317176111eda5e4c491054e8cbb07d43f56fa7b42a3ce048b914d\": container with ID starting with 04ee9653019317176111eda5e4c491054e8cbb07d43f56fa7b42a3ce048b914d not found: ID does not exist" containerID="04ee9653019317176111eda5e4c491054e8cbb07d43f56fa7b42a3ce048b914d"
Jul 13 21:41:27.448132 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:27.448081    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ee9653019317176111eda5e4c491054e8cbb07d43f56fa7b42a3ce048b914d"} err="failed to get container status \"04ee9653019317176111eda5e4c491054e8cbb07d43f56fa7b42a3ce048b914d\": rpc error: code = NotFound desc = could not find container \"04ee9653019317176111eda5e4c491054e8cbb07d43f56fa7b42a3ce048b914d\": container with ID starting with 04ee9653019317176111eda5e4c491054e8cbb07d43f56fa7b42a3ce048b914d not found: ID does not exist"
Jul 13 21:41:27.463862 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:27.463798    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4c7zl"]
Jul 13 21:41:27.465294 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:27.465271    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-4c7zl"]
Jul 13 21:41:28.472224 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:41:28.472188    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b58e70c0-96b4-43d2-887c-f3c4e748fa02" path="/var/lib/kubelet/pods/b58e70c0-96b4-43d2-887c-f3c4e748fa02/volumes"
Jul 13 21:42:31.405817 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.405779    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt"]
Jul 13 21:42:31.406234 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.406151    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b58e70c0-96b4-43d2-887c-f3c4e748fa02" containerName="authorino"
Jul 13 21:42:31.406234 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.406162    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b58e70c0-96b4-43d2-887c-f3c4e748fa02" containerName="authorino"
Jul 13 21:42:31.406234 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.406224    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b58e70c0-96b4-43d2-887c-f3c4e748fa02" containerName="authorino"
Jul 13 21:42:31.409619 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.409593    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt"
Jul 13 21:42:31.411841 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.411811    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-s2vb8\""
Jul 13 21:42:31.411841 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.411836    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 13 21:42:31.420421 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.420396    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt"]
Jul 13 21:42:31.422024 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.422000    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-9tdls"]
Jul 13 21:42:31.425317 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.425295    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-9tdls"
Jul 13 21:42:31.437431 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.437407    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-9tdls"]
Jul 13 21:42:31.473457 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.473408    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca038f40-a926-4261-bdcf-6ef05e78ae26-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-9tdls\" (UID: \"ca038f40-a926-4261-bdcf-6ef05e78ae26\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-9tdls"
Jul 13 21:42:31.473690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.473522    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t785g\" (UniqueName: \"kubernetes.io/projected/ca038f40-a926-4261-bdcf-6ef05e78ae26-kube-api-access-t785g\") pod \"payload-processing-f9b8d9b9b-9tdls\" (UID: \"ca038f40-a926-4261-bdcf-6ef05e78ae26\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-9tdls"
Jul 13 21:42:31.473690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.473581    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fefab67-ebed-402f-8f44-7553b119c7bb-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-9r9jt\" (UID: \"6fefab67-ebed-402f-8f44-7553b119c7bb\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt"
Jul 13 21:42:31.473690 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.473598    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqhth\" (UniqueName: \"kubernetes.io/projected/6fefab67-ebed-402f-8f44-7553b119c7bb-kube-api-access-xqhth\") pod \"payload-pre-processing-74584bb9f8-9r9jt\" (UID: \"6fefab67-ebed-402f-8f44-7553b119c7bb\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt"
Jul 13 21:42:31.574404 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.574364    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca038f40-a926-4261-bdcf-6ef05e78ae26-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-9tdls\" (UID: \"ca038f40-a926-4261-bdcf-6ef05e78ae26\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-9tdls"
Jul 13 21:42:31.574613 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.574534    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t785g\" (UniqueName: \"kubernetes.io/projected/ca038f40-a926-4261-bdcf-6ef05e78ae26-kube-api-access-t785g\") pod \"payload-processing-f9b8d9b9b-9tdls\" (UID: \"ca038f40-a926-4261-bdcf-6ef05e78ae26\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-9tdls"
Jul 13 21:42:31.574728 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.574635    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fefab67-ebed-402f-8f44-7553b119c7bb-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-9r9jt\" (UID: \"6fefab67-ebed-402f-8f44-7553b119c7bb\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt"
Jul 13 21:42:31.574728 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.574692    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqhth\" (UniqueName: \"kubernetes.io/projected/6fefab67-ebed-402f-8f44-7553b119c7bb-kube-api-access-xqhth\") pod \"payload-pre-processing-74584bb9f8-9r9jt\" (UID: \"6fefab67-ebed-402f-8f44-7553b119c7bb\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt"
Jul 13 21:42:31.575278 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.575253    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca038f40-a926-4261-bdcf-6ef05e78ae26-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-9tdls\" (UID: \"ca038f40-a926-4261-bdcf-6ef05e78ae26\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-9tdls"
Jul 13 21:42:31.575425 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.575410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fefab67-ebed-402f-8f44-7553b119c7bb-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-9r9jt\" (UID: \"6fefab67-ebed-402f-8f44-7553b119c7bb\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt"
Jul 13 21:42:31.582283 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.582256    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqhth\" (UniqueName: \"kubernetes.io/projected/6fefab67-ebed-402f-8f44-7553b119c7bb-kube-api-access-xqhth\") pod \"payload-pre-processing-74584bb9f8-9r9jt\" (UID: \"6fefab67-ebed-402f-8f44-7553b119c7bb\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt"
Jul 13 21:42:31.582397 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.582285    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t785g\" (UniqueName: \"kubernetes.io/projected/ca038f40-a926-4261-bdcf-6ef05e78ae26-kube-api-access-t785g\") pod \"payload-processing-f9b8d9b9b-9tdls\" (UID: \"ca038f40-a926-4261-bdcf-6ef05e78ae26\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-9tdls"
Jul 13 21:42:31.721361 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.721259    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt"
Jul 13 21:42:31.736426 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.736395    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-9tdls"
Jul 13 21:42:31.868343 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.868305    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt"]
Jul 13 21:42:31.871177 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:42:31.871149    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fefab67_ebed_402f_8f44_7553b119c7bb.slice/crio-0ac50c6cc6380defce12d94d1d85e0cabe80573c642a791e2a01bd9525715f11 WatchSource:0}: Error finding container 0ac50c6cc6380defce12d94d1d85e0cabe80573c642a791e2a01bd9525715f11: Status 404 returned error can't find the container with id 0ac50c6cc6380defce12d94d1d85e0cabe80573c642a791e2a01bd9525715f11
Jul 13 21:42:31.897953 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:31.897927    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-9tdls"]
Jul 13 21:42:31.901076 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:42:31.901040    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca038f40_a926_4261_bdcf_6ef05e78ae26.slice/crio-c4dbd8a6a5ad3250fd6345f5c700d14f1b86d297d4027aa213060444f23dc2b1 WatchSource:0}: Error finding container c4dbd8a6a5ad3250fd6345f5c700d14f1b86d297d4027aa213060444f23dc2b1: Status 404 returned error can't find the container with id c4dbd8a6a5ad3250fd6345f5c700d14f1b86d297d4027aa213060444f23dc2b1
Jul 13 21:42:32.752942 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:32.752897    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt" event={"ID":"6fefab67-ebed-402f-8f44-7553b119c7bb","Type":"ContainerStarted","Data":"0ac50c6cc6380defce12d94d1d85e0cabe80573c642a791e2a01bd9525715f11"}
Jul 13 21:42:32.754393 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:32.754353    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-9tdls" event={"ID":"ca038f40-a926-4261-bdcf-6ef05e78ae26","Type":"ContainerStarted","Data":"c4dbd8a6a5ad3250fd6345f5c700d14f1b86d297d4027aa213060444f23dc2b1"}
Jul 13 21:42:33.761187 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:33.761145    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-9tdls" event={"ID":"ca038f40-a926-4261-bdcf-6ef05e78ae26","Type":"ContainerStarted","Data":"57469a8d63c1e49f77fad7c8515e32e05c5e3b37413fbed0235fd0e9950fe994"}
Jul 13 21:42:33.761688 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:33.761219    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-9tdls"
Jul 13 21:42:33.762727 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:33.762698    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt" event={"ID":"6fefab67-ebed-402f-8f44-7553b119c7bb","Type":"ContainerStarted","Data":"072ec6ec84a985d71b4e53e002ec6a771854afbfd2699c583d8055eb3d3590f3"}
Jul 13 21:42:33.762859 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:33.762840    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt"
Jul 13 21:42:33.777835 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:33.777783    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-9tdls" podStartSLOduration=1.272469977 podStartE2EDuration="2.777766781s" totalImagesPullingTime="1.505296804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:42:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:42:33.776542346 +0000 UTC m=+795.698591263" watchObservedRunningTime="2026-07-13 21:42:33.777766781 +0000 UTC m=+795.699815698"
Jul 13 21:42:33.793031 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:33.792980    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt" podStartSLOduration=1.261041303 podStartE2EDuration="2.792952846s" totalImagesPullingTime="1.531911543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:42:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:42:33.791454765 +0000 UTC m=+795.713503705" watchObservedRunningTime="2026-07-13 21:42:33.792952846 +0000 UTC m=+795.715001763"
Jul 13 21:42:44.768971 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:44.768881    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-9tdls"
Jul 13 21:42:44.769333 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:42:44.769100    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-9r9jt"
Jul 13 21:43:07.237326 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.237291    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"]
Jul 13 21:43:07.242362 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.242340    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.244920 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.244899    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-b8vwz\""
Jul 13 21:43:07.245047 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.244946    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 13 21:43:07.245718 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.245697    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 13 21:43:07.246406 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.246374    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 13 21:43:07.248445 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.248427    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"]
Jul 13 21:43:07.295083 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.295047    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wzbr\" (UniqueName: \"kubernetes.io/projected/53fc0589-601b-4aef-b34c-53c88378d2d2-kube-api-access-9wzbr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.295265 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.295095    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53fc0589-601b-4aef-b34c-53c88378d2d2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.295265 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.295141    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53fc0589-601b-4aef-b34c-53c88378d2d2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.295265 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.295206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53fc0589-601b-4aef-b34c-53c88378d2d2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.295265 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.295225    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53fc0589-601b-4aef-b34c-53c88378d2d2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.295265 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.295254    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53fc0589-601b-4aef-b34c-53c88378d2d2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.396068 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.396029    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53fc0589-601b-4aef-b34c-53c88378d2d2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.396068 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.396070    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53fc0589-601b-4aef-b34c-53c88378d2d2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.396344 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.396099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53fc0589-601b-4aef-b34c-53c88378d2d2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.396344 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.396145    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wzbr\" (UniqueName: \"kubernetes.io/projected/53fc0589-601b-4aef-b34c-53c88378d2d2-kube-api-access-9wzbr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.396344 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.396185    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53fc0589-601b-4aef-b34c-53c88378d2d2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.396344 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.396215    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53fc0589-601b-4aef-b34c-53c88378d2d2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.396519 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.396494    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53fc0589-601b-4aef-b34c-53c88378d2d2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.396570 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.396554    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53fc0589-601b-4aef-b34c-53c88378d2d2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.396616 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.396598    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53fc0589-601b-4aef-b34c-53c88378d2d2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.398521 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.398492    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53fc0589-601b-4aef-b34c-53c88378d2d2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.398784 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.398767    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53fc0589-601b-4aef-b34c-53c88378d2d2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.404595 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.404567    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wzbr\" (UniqueName: \"kubernetes.io/projected/53fc0589-601b-4aef-b34c-53c88378d2d2-kube-api-access-9wzbr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5h6nl\" (UID: \"53fc0589-601b-4aef-b34c-53c88378d2d2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.553678 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.553627    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:07.695934 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.695910    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"]
Jul 13 21:43:07.698099 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:43:07.698073    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53fc0589_601b_4aef_b34c_53c88378d2d2.slice/crio-873d32bc6eb68083b970ce11373cd88bfa21d3b8e3ce57fe10fc72a639c7c146 WatchSource:0}: Error finding container 873d32bc6eb68083b970ce11373cd88bfa21d3b8e3ce57fe10fc72a639c7c146: Status 404 returned error can't find the container with id 873d32bc6eb68083b970ce11373cd88bfa21d3b8e3ce57fe10fc72a639c7c146
Jul 13 21:43:07.923210 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:07.923084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl" event={"ID":"53fc0589-601b-4aef-b34c-53c88378d2d2","Type":"ContainerStarted","Data":"873d32bc6eb68083b970ce11373cd88bfa21d3b8e3ce57fe10fc72a639c7c146"}
Jul 13 21:43:08.930285 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:08.930244    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl" event={"ID":"53fc0589-601b-4aef-b34c-53c88378d2d2","Type":"ContainerStarted","Data":"787cc63f89d553a1876cf3cae05b0ed05f139e59d79079eed66b59e64d5635e5"}
Jul 13 21:43:08.949151 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:08.949052    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl" podStartSLOduration=0.903591931 podStartE2EDuration="1.949029709s" totalImagesPullingTime="1.045437778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:43:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:43:08.948701578 +0000 UTC m=+830.870750495" watchObservedRunningTime="2026-07-13 21:43:08.949029709 +0000 UTC m=+830.871078627"
Jul 13 21:43:17.554585 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:17.554543    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:17.555079 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:17.554608    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:17.567051 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:17.567020    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:17.985126 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:17.985042    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5h6nl"
Jul 13 21:43:27.132949 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.132905    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"]
Jul 13 21:43:27.144303 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.144269    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.144462 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.144370    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"]
Jul 13 21:43:27.146804 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.146780    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 13 21:43:27.266514 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.266475    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.266741 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.266539    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.266741 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.266599    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.266741 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.266640    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8rxr\" (UniqueName: \"kubernetes.io/projected/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-kube-api-access-s8rxr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.266869 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.266760    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.266869 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.266806    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.367263 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.367230    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.367475 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.367281    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.367475 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.367317    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.367475 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.367333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.367475 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.367350    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8rxr\" (UniqueName: \"kubernetes.io/projected/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-kube-api-access-s8rxr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.367475 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.367390    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.367784 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.367752    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.367828 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.367791    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.367828 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.367810    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.369714 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.369685    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.369989 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.369972    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.374599 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.374570    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8rxr\" (UniqueName: \"kubernetes.io/projected/e7f3e02f-5cf2-4e0b-9213-cd2734c0f137-kube-api-access-s8rxr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zc425\" (UID: \"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.455348 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.455250    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:27.596212 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:27.596187    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"]
Jul 13 21:43:27.598529 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:43:27.598502    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7f3e02f_5cf2_4e0b_9213_cd2734c0f137.slice/crio-d25267625ea0b4313d7c58f0a1587db2eb71236eb9641a7cd5430ca8e7fee607 WatchSource:0}: Error finding container d25267625ea0b4313d7c58f0a1587db2eb71236eb9641a7cd5430ca8e7fee607: Status 404 returned error can't find the container with id d25267625ea0b4313d7c58f0a1587db2eb71236eb9641a7cd5430ca8e7fee607
Jul 13 21:43:28.024363 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:28.024267    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425" event={"ID":"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137","Type":"ContainerStarted","Data":"fb7599768fcefe7139ac06d3e37c7247c1eddcc556c3bd915ba2c46465c4406e"}
Jul 13 21:43:28.024363 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:28.024322    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425" event={"ID":"e7f3e02f-5cf2-4e0b-9213-cd2734c0f137","Type":"ContainerStarted","Data":"d25267625ea0b4313d7c58f0a1587db2eb71236eb9641a7cd5430ca8e7fee607"}
Jul 13 21:43:28.041553 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:28.041499    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425" podStartSLOduration=0.8667341 podStartE2EDuration="1.041482959s" totalImagesPullingTime="174.748859ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:43:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:43:28.040983238 +0000 UTC m=+849.963032156" watchObservedRunningTime="2026-07-13 21:43:28.041482959 +0000 UTC m=+849.963531875"
Jul 13 21:43:37.455856 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:37.455810    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:37.456278 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:37.455892    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:37.468626 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:37.468594    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:38.084871 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:38.084841    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zc425"
Jul 13 21:43:42.829925 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:42.829895    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"]
Jul 13 21:43:42.836745 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:42.836720    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:42.839074 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:42.839051    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 13 21:43:42.843115 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:42.843092    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"]
Jul 13 21:43:42.902581 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:42.902547    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5qkf\" (UniqueName: \"kubernetes.io/projected/256bfe03-ee61-4250-b399-bee682e8e28d-kube-api-access-l5qkf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:42.902767 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:42.902595    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/256bfe03-ee61-4250-b399-bee682e8e28d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:42.902767 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:42.902626    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/256bfe03-ee61-4250-b399-bee682e8e28d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:42.902767 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:42.902664    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/256bfe03-ee61-4250-b399-bee682e8e28d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:42.902767 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:42.902728    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/256bfe03-ee61-4250-b399-bee682e8e28d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:42.902903 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:42.902794    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/256bfe03-ee61-4250-b399-bee682e8e28d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.003742 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.003707    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/256bfe03-ee61-4250-b399-bee682e8e28d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.003936 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.003753    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5qkf\" (UniqueName: \"kubernetes.io/projected/256bfe03-ee61-4250-b399-bee682e8e28d-kube-api-access-l5qkf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.003936 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.003777    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/256bfe03-ee61-4250-b399-bee682e8e28d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.003936 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.003795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/256bfe03-ee61-4250-b399-bee682e8e28d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.003936 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.003813    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/256bfe03-ee61-4250-b399-bee682e8e28d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.003936 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.003906    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/256bfe03-ee61-4250-b399-bee682e8e28d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.004208 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.004147    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/256bfe03-ee61-4250-b399-bee682e8e28d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.004255 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.004192    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/256bfe03-ee61-4250-b399-bee682e8e28d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.004302 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.004277    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/256bfe03-ee61-4250-b399-bee682e8e28d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.006094 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.006068    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/256bfe03-ee61-4250-b399-bee682e8e28d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.006383 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.006364    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/256bfe03-ee61-4250-b399-bee682e8e28d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.011532 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.011510    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5qkf\" (UniqueName: \"kubernetes.io/projected/256bfe03-ee61-4250-b399-bee682e8e28d-kube-api-access-l5qkf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7\" (UID: \"256bfe03-ee61-4250-b399-bee682e8e28d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.147909 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.147817    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:43.290176 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:43.290023    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"]
Jul 13 21:43:43.292621 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:43:43.292594    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod256bfe03_ee61_4250_b399_bee682e8e28d.slice/crio-1ca974f9cd848202567fcbe34a4e21b2c4ac8bd19b3097cddf2de5d16ef69f7e WatchSource:0}: Error finding container 1ca974f9cd848202567fcbe34a4e21b2c4ac8bd19b3097cddf2de5d16ef69f7e: Status 404 returned error can't find the container with id 1ca974f9cd848202567fcbe34a4e21b2c4ac8bd19b3097cddf2de5d16ef69f7e
Jul 13 21:43:44.105165 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:44.105127    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7" event={"ID":"256bfe03-ee61-4250-b399-bee682e8e28d","Type":"ContainerStarted","Data":"b1cb7a519ea10bb73e8448d95031af5ab9fe55f26ba8cd643cbbe10a5ea511c7"}
Jul 13 21:43:44.105557 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:44.105176    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7" event={"ID":"256bfe03-ee61-4250-b399-bee682e8e28d","Type":"ContainerStarted","Data":"1ca974f9cd848202567fcbe34a4e21b2c4ac8bd19b3097cddf2de5d16ef69f7e"}
Jul 13 21:43:44.125968 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:44.125917    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7" podStartSLOduration=1.919451707 podStartE2EDuration="2.125899823s" totalImagesPullingTime="206.448116ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:43:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:43:44.124917086 +0000 UTC m=+866.046966003" watchObservedRunningTime="2026-07-13 21:43:44.125899823 +0000 UTC m=+866.047948740"
Jul 13 21:43:53.148530 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:53.148489    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:53.149041 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:53.148543    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:53.160869 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:53.160835    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:43:54.166037 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:43:54.166008    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7"
Jul 13 21:44:18.437456 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:44:18.437374    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:44:18.438798 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:44:18.438779    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:49:18.468399 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:49:18.468366    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:49:18.471169 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:49:18.471148    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:50:25.692436 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:25.692353    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"]
Jul 13 21:50:25.695064 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:25.692596    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn" podUID="48474f36-7987-43b5-a9ac-e2461b54258e" containerName="manager" containerID="cri-o://3cdeb2b7de2992fd766ea82d428562a8ae74850c0f9c37bf67788a52155fe3ff" gracePeriod=10
Jul 13 21:50:28.946714 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:28.946687    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"
Jul 13 21:50:28.995958 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:28.995928    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48474f36-7987-43b5-a9ac-e2461b54258e-extensions-socket-volume\") pod \"48474f36-7987-43b5-a9ac-e2461b54258e\" (UID: \"48474f36-7987-43b5-a9ac-e2461b54258e\") "
Jul 13 21:50:28.996114 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:28.995996    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79bpd\" (UniqueName: \"kubernetes.io/projected/48474f36-7987-43b5-a9ac-e2461b54258e-kube-api-access-79bpd\") pod \"48474f36-7987-43b5-a9ac-e2461b54258e\" (UID: \"48474f36-7987-43b5-a9ac-e2461b54258e\") "
Jul 13 21:50:28.996322 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:28.996296    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48474f36-7987-43b5-a9ac-e2461b54258e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "48474f36-7987-43b5-a9ac-e2461b54258e" (UID: "48474f36-7987-43b5-a9ac-e2461b54258e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:50:28.998177 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:28.998151    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48474f36-7987-43b5-a9ac-e2461b54258e-kube-api-access-79bpd" (OuterVolumeSpecName: "kube-api-access-79bpd") pod "48474f36-7987-43b5-a9ac-e2461b54258e" (UID: "48474f36-7987-43b5-a9ac-e2461b54258e"). InnerVolumeSpecName "kube-api-access-79bpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:50:29.096829 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:29.096734    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48474f36-7987-43b5-a9ac-e2461b54258e-extensions-socket-volume\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:50:29.096829 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:29.096759    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79bpd\" (UniqueName: \"kubernetes.io/projected/48474f36-7987-43b5-a9ac-e2461b54258e-kube-api-access-79bpd\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 21:50:29.097063 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:29.097044    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"
Jul 13 21:50:29.097123 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:29.096980    2564 generic.go:356] "Generic (PLEG): container finished" podID="48474f36-7987-43b5-a9ac-e2461b54258e" containerID="3cdeb2b7de2992fd766ea82d428562a8ae74850c0f9c37bf67788a52155fe3ff" exitCode=0
Jul 13 21:50:29.097123 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:29.097069    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn" event={"ID":"48474f36-7987-43b5-a9ac-e2461b54258e","Type":"ContainerDied","Data":"3cdeb2b7de2992fd766ea82d428562a8ae74850c0f9c37bf67788a52155fe3ff"}
Jul 13 21:50:29.097220 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:29.097138    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn" event={"ID":"48474f36-7987-43b5-a9ac-e2461b54258e","Type":"ContainerDied","Data":"3f01e39015fd95f468b45e040c15a77464b540b0c53e767de7095544927c979b"}
Jul 13 21:50:29.097220 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:29.097173    2564 scope.go:117] "RemoveContainer" containerID="3cdeb2b7de2992fd766ea82d428562a8ae74850c0f9c37bf67788a52155fe3ff"
Jul 13 21:50:29.111224 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:29.111204    2564 scope.go:117] "RemoveContainer" containerID="3cdeb2b7de2992fd766ea82d428562a8ae74850c0f9c37bf67788a52155fe3ff"
Jul 13 21:50:29.111517 ip-10-0-140-98 kubenswrapper[2564]: E0713 21:50:29.111497    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cdeb2b7de2992fd766ea82d428562a8ae74850c0f9c37bf67788a52155fe3ff\": container with ID starting with 3cdeb2b7de2992fd766ea82d428562a8ae74850c0f9c37bf67788a52155fe3ff not found: ID does not exist" containerID="3cdeb2b7de2992fd766ea82d428562a8ae74850c0f9c37bf67788a52155fe3ff"
Jul 13 21:50:29.111593 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:29.111525    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cdeb2b7de2992fd766ea82d428562a8ae74850c0f9c37bf67788a52155fe3ff"} err="failed to get container status \"3cdeb2b7de2992fd766ea82d428562a8ae74850c0f9c37bf67788a52155fe3ff\": rpc error: code = NotFound desc = could not find container \"3cdeb2b7de2992fd766ea82d428562a8ae74850c0f9c37bf67788a52155fe3ff\": container with ID starting with 3cdeb2b7de2992fd766ea82d428562a8ae74850c0f9c37bf67788a52155fe3ff not found: ID does not exist"
Jul 13 21:50:29.122100 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:29.122075    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"]
Jul 13 21:50:29.125786 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:29.125762    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fvtn"]
Jul 13 21:50:30.473264 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:50:30.473232    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48474f36-7987-43b5-a9ac-e2461b54258e" path="/var/lib/kubelet/pods/48474f36-7987-43b5-a9ac-e2461b54258e/volumes"
Jul 13 21:51:26.798267 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:26.798233    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz"]
Jul 13 21:51:26.798750 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:26.798615    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48474f36-7987-43b5-a9ac-e2461b54258e" containerName="manager"
Jul 13 21:51:26.798750 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:26.798626    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="48474f36-7987-43b5-a9ac-e2461b54258e" containerName="manager"
Jul 13 21:51:26.798750 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:26.798710    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="48474f36-7987-43b5-a9ac-e2461b54258e" containerName="manager"
Jul 13 21:51:26.802976 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:26.802956    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz"
Jul 13 21:51:26.805396 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:26.805374    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 21:51:26.805567 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:26.805495    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v6jfl\""
Jul 13 21:51:26.805567 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:26.805495    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 21:51:26.811984 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:26.811961    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz"]
Jul 13 21:51:26.917319 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:26.917287    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kgdd\" (UniqueName: \"kubernetes.io/projected/8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8-kube-api-access-5kgdd\") pod \"kuadrant-operator-controller-manager-86878c769f-jzdzz\" (UID: \"8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz"
Jul 13 21:51:26.917319 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:26.917333    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jzdzz\" (UID: \"8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz"
Jul 13 21:51:27.018439 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:27.018402    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kgdd\" (UniqueName: \"kubernetes.io/projected/8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8-kube-api-access-5kgdd\") pod \"kuadrant-operator-controller-manager-86878c769f-jzdzz\" (UID: \"8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz"
Jul 13 21:51:27.018578 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:27.018452    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jzdzz\" (UID: \"8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz"
Jul 13 21:51:27.018842 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:27.018825    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jzdzz\" (UID: \"8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz"
Jul 13 21:51:27.027589 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:27.027561    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kgdd\" (UniqueName: \"kubernetes.io/projected/8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8-kube-api-access-5kgdd\") pod \"kuadrant-operator-controller-manager-86878c769f-jzdzz\" (UID: \"8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz"
Jul 13 21:51:27.115295 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:27.115207    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz"
Jul 13 21:51:27.257305 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:27.257280    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz"]
Jul 13 21:51:27.260302 ip-10-0-140-98 kubenswrapper[2564]: W0713 21:51:27.260269    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c2a9b5e_e8c6_494c_a98d_bca8bd605ad8.slice/crio-31801f0e4f33084a68d42bf7ae0863c653be3512c5c6d1bae7963b57dc94cae6 WatchSource:0}: Error finding container 31801f0e4f33084a68d42bf7ae0863c653be3512c5c6d1bae7963b57dc94cae6: Status 404 returned error can't find the container with id 31801f0e4f33084a68d42bf7ae0863c653be3512c5c6d1bae7963b57dc94cae6
Jul 13 21:51:27.381746 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:27.381629    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz" event={"ID":"8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8","Type":"ContainerStarted","Data":"00764f6723160c26495a5e63d535829668d45437f3e872c4c03f3996e7fb14c8"}
Jul 13 21:51:27.381746 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:27.381712    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz" event={"ID":"8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8","Type":"ContainerStarted","Data":"31801f0e4f33084a68d42bf7ae0863c653be3512c5c6d1bae7963b57dc94cae6"}
Jul 13 21:51:27.381746 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:27.381746    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz"
Jul 13 21:51:27.401871 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:27.401824    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz" podStartSLOduration=1.401806838 podStartE2EDuration="1.401806838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:51:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:51:27.399871223 +0000 UTC m=+1329.321920141" watchObservedRunningTime="2026-07-13 21:51:27.401806838 +0000 UTC m=+1329.323855756"
Jul 13 21:51:38.389006 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:51:38.388974    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jzdzz"
Jul 13 21:54:18.501126 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:54:18.501092    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:54:18.504175 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:54:18.504152    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:59:18.532423 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:59:18.532394    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 21:59:18.535204 ip-10-0-140-98 kubenswrapper[2564]: I0713 21:59:18.535183    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 22:04:18.561979 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:04:18.561949    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 22:04:18.565924 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:04:18.565895    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 22:06:02.835116 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.835083    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"]
Jul 13 22:06:02.838730 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.838708    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.841495 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.841467    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-06fcd095\"/\"kube-root-ca.crt\""
Jul 13 22:06:02.841640 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.841626    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-06fcd095\"/\"openshift-service-ca.crt\""
Jul 13 22:06:02.842541 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.842523    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-06fcd095\"/\"default-dockercfg-8vkr8\""
Jul 13 22:06:02.842635 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.842586    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-06fcd095\"/\"e2e-lifecycle-model-06fcd095-kserve-self-signed-certs\""
Jul 13 22:06:02.847664 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.847629    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"]
Jul 13 22:06:02.876417 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.876387    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5354841-c005-4ee2-bf39-2b78f338ddd5-tls-certs\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.876417 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.876420    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-dshm\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.876616 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.876444    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-tmp-dir\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.876616 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.876528    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-model-cache\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.876616 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.876572    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-home\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.876616 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.876602    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wp7z\" (UniqueName: \"kubernetes.io/projected/e5354841-c005-4ee2-bf39-2b78f338ddd5-kube-api-access-7wp7z\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.977452 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.977413    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5354841-c005-4ee2-bf39-2b78f338ddd5-tls-certs\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.977452 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.977456    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-dshm\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.977731 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.977481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-tmp-dir\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.977731 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.977533    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-model-cache\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.977731 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.977577    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-home\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.977731 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.977615    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wp7z\" (UniqueName: \"kubernetes.io/projected/e5354841-c005-4ee2-bf39-2b78f338ddd5-kube-api-access-7wp7z\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.978388 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.978008    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-tmp-dir\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.978388 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.978042    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-model-cache\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.978388 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.978059    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-home\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.979905 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.979868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-dshm\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.980141 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.980122    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5354841-c005-4ee2-bf39-2b78f338ddd5-tls-certs\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:02.986490 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:02.986464    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wp7z\" (UniqueName: \"kubernetes.io/projected/e5354841-c005-4ee2-bf39-2b78f338ddd5-kube-api-access-7wp7z\") pod \"e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") " pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:03.149063 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:03.148959    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:03.290520 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:03.290485    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"]
Jul 13 22:06:03.293317 ip-10-0-140-98 kubenswrapper[2564]: W0713 22:06:03.293278    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5354841_c005_4ee2_bf39_2b78f338ddd5.slice/crio-0fcf7cd408f6aeacea2491226ea1a95e39e08b993adb52e4dc7e19cd18c3c5ef WatchSource:0}: Error finding container 0fcf7cd408f6aeacea2491226ea1a95e39e08b993adb52e4dc7e19cd18c3c5ef: Status 404 returned error can't find the container with id 0fcf7cd408f6aeacea2491226ea1a95e39e08b993adb52e4dc7e19cd18c3c5ef
Jul 13 22:06:03.669183 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:03.669142    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j" event={"ID":"e5354841-c005-4ee2-bf39-2b78f338ddd5","Type":"ContainerStarted","Data":"d7e81c473bbe437219b21bfa741b76fc91749b2a7f2b1e86a696ba0138e781b0"}
Jul 13 22:06:03.669360 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:03.669191    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j" event={"ID":"e5354841-c005-4ee2-bf39-2b78f338ddd5","Type":"ContainerStarted","Data":"0fcf7cd408f6aeacea2491226ea1a95e39e08b993adb52e4dc7e19cd18c3c5ef"}
Jul 13 22:06:03.691434 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:03.691381    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j" podStartSLOduration=1.6913648110000001 podStartE2EDuration="1.691364811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:06:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:06:03.689284302 +0000 UTC m=+2205.611333230" watchObservedRunningTime="2026-07-13 22:06:03.691364811 +0000 UTC m=+2205.613413727"
Jul 13 22:06:13.149225 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:13.149187    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:13.149747 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:13.149239    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:13.162142 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:13.162115    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:13.733486 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:13.733454    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:41.760510 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:41.760430    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"]
Jul 13 22:06:56.782047 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:56.781977    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j" podUID="e5354841-c005-4ee2-bf39-2b78f338ddd5" containerName="main" containerID="cri-o://d7e81c473bbe437219b21bfa741b76fc91749b2a7f2b1e86a696ba0138e781b0" gracePeriod=45
Jul 13 22:06:56.947260 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:56.947220    2564 generic.go:356] "Generic (PLEG): container finished" podID="e5354841-c005-4ee2-bf39-2b78f338ddd5" containerID="d7e81c473bbe437219b21bfa741b76fc91749b2a7f2b1e86a696ba0138e781b0" exitCode=0
Jul 13 22:06:56.947478 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:56.947293    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j" event={"ID":"e5354841-c005-4ee2-bf39-2b78f338ddd5","Type":"ContainerDied","Data":"d7e81c473bbe437219b21bfa741b76fc91749b2a7f2b1e86a696ba0138e781b0"}
Jul 13 22:06:57.040599 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.040530    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:57.076805 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.076776    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wp7z\" (UniqueName: \"kubernetes.io/projected/e5354841-c005-4ee2-bf39-2b78f338ddd5-kube-api-access-7wp7z\") pod \"e5354841-c005-4ee2-bf39-2b78f338ddd5\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") "
Jul 13 22:06:57.077002 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.076858    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-tmp-dir\") pod \"e5354841-c005-4ee2-bf39-2b78f338ddd5\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") "
Jul 13 22:06:57.077002 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.076884    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-dshm\") pod \"e5354841-c005-4ee2-bf39-2b78f338ddd5\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") "
Jul 13 22:06:57.077002 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.076922    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-model-cache\") pod \"e5354841-c005-4ee2-bf39-2b78f338ddd5\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") "
Jul 13 22:06:57.077002 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.076947    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5354841-c005-4ee2-bf39-2b78f338ddd5-tls-certs\") pod \"e5354841-c005-4ee2-bf39-2b78f338ddd5\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") "
Jul 13 22:06:57.077002 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.076976    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-home\") pod \"e5354841-c005-4ee2-bf39-2b78f338ddd5\" (UID: \"e5354841-c005-4ee2-bf39-2b78f338ddd5\") "
Jul 13 22:06:57.077244 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.077189    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e5354841-c005-4ee2-bf39-2b78f338ddd5" (UID: "e5354841-c005-4ee2-bf39-2b78f338ddd5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:06:57.077244 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.077197    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-model-cache" (OuterVolumeSpecName: "model-cache") pod "e5354841-c005-4ee2-bf39-2b78f338ddd5" (UID: "e5354841-c005-4ee2-bf39-2b78f338ddd5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:06:57.077352 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.077303    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-tmp-dir\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:06:57.077352 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.077322    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-model-cache\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:06:57.077457 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.077429    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-home" (OuterVolumeSpecName: "home") pod "e5354841-c005-4ee2-bf39-2b78f338ddd5" (UID: "e5354841-c005-4ee2-bf39-2b78f338ddd5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:06:57.079310 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.079282    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5354841-c005-4ee2-bf39-2b78f338ddd5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e5354841-c005-4ee2-bf39-2b78f338ddd5" (UID: "e5354841-c005-4ee2-bf39-2b78f338ddd5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:06:57.079535 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.079504    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5354841-c005-4ee2-bf39-2b78f338ddd5-kube-api-access-7wp7z" (OuterVolumeSpecName: "kube-api-access-7wp7z") pod "e5354841-c005-4ee2-bf39-2b78f338ddd5" (UID: "e5354841-c005-4ee2-bf39-2b78f338ddd5"). InnerVolumeSpecName "kube-api-access-7wp7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:06:57.079535 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.079504    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-dshm" (OuterVolumeSpecName: "dshm") pod "e5354841-c005-4ee2-bf39-2b78f338ddd5" (UID: "e5354841-c005-4ee2-bf39-2b78f338ddd5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:06:57.178805 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.178769    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wp7z\" (UniqueName: \"kubernetes.io/projected/e5354841-c005-4ee2-bf39-2b78f338ddd5-kube-api-access-7wp7z\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:06:57.178805 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.178799    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-dshm\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:06:57.178805 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.178808    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5354841-c005-4ee2-bf39-2b78f338ddd5-tls-certs\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:06:57.178805 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.178817    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5354841-c005-4ee2-bf39-2b78f338ddd5-home\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:06:57.953398 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.953360    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j" event={"ID":"e5354841-c005-4ee2-bf39-2b78f338ddd5","Type":"ContainerDied","Data":"0fcf7cd408f6aeacea2491226ea1a95e39e08b993adb52e4dc7e19cd18c3c5ef"}
Jul 13 22:06:57.953916 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.953409    2564 scope.go:117] "RemoveContainer" containerID="d7e81c473bbe437219b21bfa741b76fc91749b2a7f2b1e86a696ba0138e781b0"
Jul 13 22:06:57.953916 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.953376    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"
Jul 13 22:06:57.975936 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.975896    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"]
Jul 13 22:06:57.981311 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:57.981286    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-06fcd095/e2e-lifecycle-model-06fcd095-kserve-767cbcd6bc-f8j9j"]
Jul 13 22:06:58.473433 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:06:58.473397    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5354841-c005-4ee2-bf39-2b78f338ddd5" path="/var/lib/kubelet/pods/e5354841-c005-4ee2-bf39-2b78f338ddd5/volumes"
Jul 13 22:09:15.540411 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.540378    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"]
Jul 13 22:09:15.540851 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.540785    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5354841-c005-4ee2-bf39-2b78f338ddd5" containerName="main"
Jul 13 22:09:15.540851 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.540799    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5354841-c005-4ee2-bf39-2b78f338ddd5" containerName="main"
Jul 13 22:09:15.540929 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.540862    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5354841-c005-4ee2-bf39-2b78f338ddd5" containerName="main"
Jul 13 22:09:15.543980 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.543961    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.546971 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.546939    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-160510\"/\"kube-root-ca.crt\""
Jul 13 22:09:15.547591 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.547567    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-160510\"/\"openshift-service-ca.crt\""
Jul 13 22:09:15.547738 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.547617    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-160510\"/\"default-dockercfg-bqmd9\""
Jul 13 22:09:15.547738 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.547636    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-160510\"/\"test-model-160510-kserve-self-signed-certs\""
Jul 13 22:09:15.553322 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.553301    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"]
Jul 13 22:09:15.634164 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.634119    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33797694-b0bf-48bc-bb2f-1aa3191dbe02-tls-certs\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.634164 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.634165    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-tmp-dir\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.634444 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.634187    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-home\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.634444 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.634217    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-dshm\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.634444 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.634233    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckvxx\" (UniqueName: \"kubernetes.io/projected/33797694-b0bf-48bc-bb2f-1aa3191dbe02-kube-api-access-ckvxx\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.634444 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.634283    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-model-cache\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.734948 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.734915    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33797694-b0bf-48bc-bb2f-1aa3191dbe02-tls-certs\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.734948 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.734958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-tmp-dir\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.735223 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.734988    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-home\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.735223 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.735039    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-dshm\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.735223 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.735061    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckvxx\" (UniqueName: \"kubernetes.io/projected/33797694-b0bf-48bc-bb2f-1aa3191dbe02-kube-api-access-ckvxx\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.735223 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.735190    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-model-cache\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.735415 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.735394    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-tmp-dir\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.735482 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.735444    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-home\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.735554 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.735529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-model-cache\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.737347 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.737318    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-dshm\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.737628 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.737610    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33797694-b0bf-48bc-bb2f-1aa3191dbe02-tls-certs\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.743800 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.743773    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckvxx\" (UniqueName: \"kubernetes.io/projected/33797694-b0bf-48bc-bb2f-1aa3191dbe02-kube-api-access-ckvxx\") pod \"test-model-160510-kserve-c66ddd4dc-7z8n7\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") " pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:15.854551 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:15.854463    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:16.002289 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:16.002187    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"]
Jul 13 22:09:16.005275 ip-10-0-140-98 kubenswrapper[2564]: W0713 22:09:16.005240    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33797694_b0bf_48bc_bb2f_1aa3191dbe02.slice/crio-9776e91e907b44d5c1110fac8381a348d84afc768f0f56aa14a4c080de213034 WatchSource:0}: Error finding container 9776e91e907b44d5c1110fac8381a348d84afc768f0f56aa14a4c080de213034: Status 404 returned error can't find the container with id 9776e91e907b44d5c1110fac8381a348d84afc768f0f56aa14a4c080de213034
Jul 13 22:09:16.645328 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:16.645279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7" event={"ID":"33797694-b0bf-48bc-bb2f-1aa3191dbe02","Type":"ContainerStarted","Data":"85fd79c7450c62247386e5b40839ae99dd7bf990e22da9f0d745a5e1b87b7479"}
Jul 13 22:09:16.645328 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:16.645330    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7" event={"ID":"33797694-b0bf-48bc-bb2f-1aa3191dbe02","Type":"ContainerStarted","Data":"9776e91e907b44d5c1110fac8381a348d84afc768f0f56aa14a4c080de213034"}
Jul 13 22:09:16.671376 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:16.671318    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7" podStartSLOduration=1.671301218 podStartE2EDuration="1.671301218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:09:16.669510855 +0000 UTC m=+2398.591559773" watchObservedRunningTime="2026-07-13 22:09:16.671301218 +0000 UTC m=+2398.593350135"
Jul 13 22:09:18.593382 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:18.593349    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 22:09:18.599681 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:18.599636    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 22:09:25.854848 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:25.854801    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:25.855232 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:25.854859    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:25.867593 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:25.867551    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:26.706440 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:26.706411    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:09:52.985595 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:09:52.985507    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"]
Jul 13 22:10:08.006464 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.006415    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7" podUID="33797694-b0bf-48bc-bb2f-1aa3191dbe02" containerName="main" containerID="cri-o://85fd79c7450c62247386e5b40839ae99dd7bf990e22da9f0d745a5e1b87b7479" gracePeriod=45
Jul 13 22:10:08.266683 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.266584    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:10:08.293841 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.293802    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-dshm\") pod \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") "
Jul 13 22:10:08.294032 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.293898    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33797694-b0bf-48bc-bb2f-1aa3191dbe02-tls-certs\") pod \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") "
Jul 13 22:10:08.294032 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.293938    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-model-cache\") pod \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") "
Jul 13 22:10:08.294032 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.293965    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-tmp-dir\") pod \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") "
Jul 13 22:10:08.294032 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.293990    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckvxx\" (UniqueName: \"kubernetes.io/projected/33797694-b0bf-48bc-bb2f-1aa3191dbe02-kube-api-access-ckvxx\") pod \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") "
Jul 13 22:10:08.294262 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.294068    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-home\") pod \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\" (UID: \"33797694-b0bf-48bc-bb2f-1aa3191dbe02\") "
Jul 13 22:10:08.294536 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.294389    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "33797694-b0bf-48bc-bb2f-1aa3191dbe02" (UID: "33797694-b0bf-48bc-bb2f-1aa3191dbe02"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:10:08.294536 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.294499    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-model-cache" (OuterVolumeSpecName: "model-cache") pod "33797694-b0bf-48bc-bb2f-1aa3191dbe02" (UID: "33797694-b0bf-48bc-bb2f-1aa3191dbe02"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:10:08.294536 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.294515    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-home" (OuterVolumeSpecName: "home") pod "33797694-b0bf-48bc-bb2f-1aa3191dbe02" (UID: "33797694-b0bf-48bc-bb2f-1aa3191dbe02"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:10:08.296362 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.296329    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33797694-b0bf-48bc-bb2f-1aa3191dbe02-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "33797694-b0bf-48bc-bb2f-1aa3191dbe02" (UID: "33797694-b0bf-48bc-bb2f-1aa3191dbe02"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:10:08.296633 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.296605    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-dshm" (OuterVolumeSpecName: "dshm") pod "33797694-b0bf-48bc-bb2f-1aa3191dbe02" (UID: "33797694-b0bf-48bc-bb2f-1aa3191dbe02"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:10:08.296817 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.296799    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33797694-b0bf-48bc-bb2f-1aa3191dbe02-kube-api-access-ckvxx" (OuterVolumeSpecName: "kube-api-access-ckvxx") pod "33797694-b0bf-48bc-bb2f-1aa3191dbe02" (UID: "33797694-b0bf-48bc-bb2f-1aa3191dbe02"). InnerVolumeSpecName "kube-api-access-ckvxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:10:08.395350 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.395308    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckvxx\" (UniqueName: \"kubernetes.io/projected/33797694-b0bf-48bc-bb2f-1aa3191dbe02-kube-api-access-ckvxx\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:10:08.395350 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.395339    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-home\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:10:08.395350 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.395351    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-dshm\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:10:08.395350 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.395361    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33797694-b0bf-48bc-bb2f-1aa3191dbe02-tls-certs\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:10:08.395741 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.395371    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-model-cache\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:10:08.395741 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.395378    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33797694-b0bf-48bc-bb2f-1aa3191dbe02-tmp-dir\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:10:08.905598 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.905562    2564 generic.go:356] "Generic (PLEG): container finished" podID="33797694-b0bf-48bc-bb2f-1aa3191dbe02" containerID="85fd79c7450c62247386e5b40839ae99dd7bf990e22da9f0d745a5e1b87b7479" exitCode=0
Jul 13 22:10:08.905805 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.905608    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"
Jul 13 22:10:08.905805 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.905664    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7" event={"ID":"33797694-b0bf-48bc-bb2f-1aa3191dbe02","Type":"ContainerDied","Data":"85fd79c7450c62247386e5b40839ae99dd7bf990e22da9f0d745a5e1b87b7479"}
Jul 13 22:10:08.905805 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.905713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7" event={"ID":"33797694-b0bf-48bc-bb2f-1aa3191dbe02","Type":"ContainerDied","Data":"9776e91e907b44d5c1110fac8381a348d84afc768f0f56aa14a4c080de213034"}
Jul 13 22:10:08.905805 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.905736    2564 scope.go:117] "RemoveContainer" containerID="85fd79c7450c62247386e5b40839ae99dd7bf990e22da9f0d745a5e1b87b7479"
Jul 13 22:10:08.914619 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.914597    2564 scope.go:117] "RemoveContainer" containerID="85fd79c7450c62247386e5b40839ae99dd7bf990e22da9f0d745a5e1b87b7479"
Jul 13 22:10:08.914903 ip-10-0-140-98 kubenswrapper[2564]: E0713 22:10:08.914881    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85fd79c7450c62247386e5b40839ae99dd7bf990e22da9f0d745a5e1b87b7479\": container with ID starting with 85fd79c7450c62247386e5b40839ae99dd7bf990e22da9f0d745a5e1b87b7479 not found: ID does not exist" containerID="85fd79c7450c62247386e5b40839ae99dd7bf990e22da9f0d745a5e1b87b7479"
Jul 13 22:10:08.914972 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.914915    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85fd79c7450c62247386e5b40839ae99dd7bf990e22da9f0d745a5e1b87b7479"} err="failed to get container status \"85fd79c7450c62247386e5b40839ae99dd7bf990e22da9f0d745a5e1b87b7479\": rpc error: code = NotFound desc = could not find container \"85fd79c7450c62247386e5b40839ae99dd7bf990e22da9f0d745a5e1b87b7479\": container with ID starting with 85fd79c7450c62247386e5b40839ae99dd7bf990e22da9f0d745a5e1b87b7479 not found: ID does not exist"
Jul 13 22:10:08.938337 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.938312    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"]
Jul 13 22:10:08.943082 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:08.943063    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-160510/test-model-160510-kserve-c66ddd4dc-7z8n7"]
Jul 13 22:10:10.473837 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:10:10.473806    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33797694-b0bf-48bc-bb2f-1aa3191dbe02" path="/var/lib/kubelet/pods/33797694-b0bf-48bc-bb2f-1aa3191dbe02/volumes"
Jul 13 22:12:33.540208 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.540172    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"]
Jul 13 22:12:33.540729 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.540557    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33797694-b0bf-48bc-bb2f-1aa3191dbe02" containerName="main"
Jul 13 22:12:33.540729 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.540569    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="33797694-b0bf-48bc-bb2f-1aa3191dbe02" containerName="main"
Jul 13 22:12:33.540729 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.540677    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="33797694-b0bf-48bc-bb2f-1aa3191dbe02" containerName="main"
Jul 13 22:12:33.543943 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.543917    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.546349 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.546326    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f89343\"/\"kube-root-ca.crt\""
Jul 13 22:12:33.547366 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.547348    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f89343\"/\"openshift-service-ca.crt\""
Jul 13 22:12:33.547366 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.547361    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f89343\"/\"auth-test-model-f89343-kserve-self-signed-certs\""
Jul 13 22:12:33.547556 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.547363    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f89343\"/\"default-dockercfg-sw9d8\""
Jul 13 22:12:33.551510 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.551479    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"]
Jul 13 22:12:33.701706 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.701625    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-home\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.701706 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.701689    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-tmp-dir\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.701983 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.701735    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2t6j\" (UniqueName: \"kubernetes.io/projected/dd25585c-450c-4bd7-b8fe-6fc466044baa-kube-api-access-n2t6j\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.701983 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.701859    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-model-cache\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.701983 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.701888    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-dshm\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.701983 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.701977    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd25585c-450c-4bd7-b8fe-6fc466044baa-tls-certs\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.802886 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.802790    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd25585c-450c-4bd7-b8fe-6fc466044baa-tls-certs\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.802886 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.802847    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-home\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.802886 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.802871    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-tmp-dir\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.803160 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.802894    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2t6j\" (UniqueName: \"kubernetes.io/projected/dd25585c-450c-4bd7-b8fe-6fc466044baa-kube-api-access-n2t6j\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.803160 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.802956    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-model-cache\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.803160 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.803093    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-dshm\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.803319 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.803303    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-home\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.803380 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.803341    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-tmp-dir\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.803436 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.803378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-model-cache\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.805324 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.805297    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-dshm\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.805480 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.805463    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd25585c-450c-4bd7-b8fe-6fc466044baa-tls-certs\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.811438 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.811406    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2t6j\" (UniqueName: \"kubernetes.io/projected/dd25585c-450c-4bd7-b8fe-6fc466044baa-kube-api-access-n2t6j\") pod \"auth-test-model-f89343-kserve-76df657967-lstml\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") " pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:33.855013 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:33.854971    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:34.022149 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:34.022122    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"]
Jul 13 22:12:34.024385 ip-10-0-140-98 kubenswrapper[2564]: W0713 22:12:34.024350    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd25585c_450c_4bd7_b8fe_6fc466044baa.slice/crio-54f5ce09bbc65ae98a5418841abd6b866d72b877754637517fab8f7b81a44a40 WatchSource:0}: Error finding container 54f5ce09bbc65ae98a5418841abd6b866d72b877754637517fab8f7b81a44a40: Status 404 returned error can't find the container with id 54f5ce09bbc65ae98a5418841abd6b866d72b877754637517fab8f7b81a44a40
Jul 13 22:12:34.622574 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:34.622535    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml" event={"ID":"dd25585c-450c-4bd7-b8fe-6fc466044baa","Type":"ContainerStarted","Data":"1bbe08717d1064bda7af61ef7890d230b0539460edd9aaaced27a05e03ba34a9"}
Jul 13 22:12:34.622574 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:34.622579    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml" event={"ID":"dd25585c-450c-4bd7-b8fe-6fc466044baa","Type":"ContainerStarted","Data":"54f5ce09bbc65ae98a5418841abd6b866d72b877754637517fab8f7b81a44a40"}
Jul 13 22:12:34.641701 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:34.641616    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml" podStartSLOduration=1.641595491 podStartE2EDuration="1.641595491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:12:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:12:34.640587776 +0000 UTC m=+2596.562636695" watchObservedRunningTime="2026-07-13 22:12:34.641595491 +0000 UTC m=+2596.563644409"
Jul 13 22:12:43.855533 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:43.855424    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:43.855533 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:43.855486    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:43.868308 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:43.868281    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:12:44.681809 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:12:44.681775    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:14:18.624671 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:14:18.624577    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 22:14:18.638105 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:14:18.638077    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 22:14:52.718709 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:14:52.718667    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"]
Jul 13 22:15:02.632834 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.632796    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"]
Jul 13 22:15:02.636291 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.636274    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.638548 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.638528    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f89343\"/\"sub-test-model-f89343-kserve-self-signed-certs\""
Jul 13 22:15:02.643609 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.643375    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"]
Jul 13 22:15:02.670267 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.670228    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/504278ff-33da-4193-98fe-70bd79e0d35b-tls-certs\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.670484 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.670276    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkw64\" (UniqueName: \"kubernetes.io/projected/504278ff-33da-4193-98fe-70bd79e0d35b-kube-api-access-dkw64\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.670484 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.670339    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-home\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.670484 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.670378    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-tmp-dir\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.670484 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.670451    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-dshm\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.670484 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.670474    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-model-cache\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.771604 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.771556    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-home\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.771604 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.771605    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-tmp-dir\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.771896 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.771750    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-dshm\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.771896 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.771771    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-model-cache\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.771896 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.771851    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/504278ff-33da-4193-98fe-70bd79e0d35b-tls-certs\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.771896 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.771887    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkw64\" (UniqueName: \"kubernetes.io/projected/504278ff-33da-4193-98fe-70bd79e0d35b-kube-api-access-dkw64\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.772133 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.772070    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-home\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.772186 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.772128    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-tmp-dir\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.772227 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.772205    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-model-cache\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.774225 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.774194    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-dshm\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.774459 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.774440    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/504278ff-33da-4193-98fe-70bd79e0d35b-tls-certs\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.780544 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.780519    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkw64\" (UniqueName: \"kubernetes.io/projected/504278ff-33da-4193-98fe-70bd79e0d35b-kube-api-access-dkw64\") pod \"sub-test-model-f89343-kserve-6f46cf768f-lv4zb\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") " pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:02.946311 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:02.946208    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:03.086053 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:03.086028    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"]
Jul 13 22:15:03.088306 ip-10-0-140-98 kubenswrapper[2564]: W0713 22:15:03.088276    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod504278ff_33da_4193_98fe_70bd79e0d35b.slice/crio-ad939decfbde48a45c55d33d944c018a5fea26ec563364956930222c1d7190a9 WatchSource:0}: Error finding container ad939decfbde48a45c55d33d944c018a5fea26ec563364956930222c1d7190a9: Status 404 returned error can't find the container with id ad939decfbde48a45c55d33d944c018a5fea26ec563364956930222c1d7190a9
Jul 13 22:15:03.364143 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:03.364104    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb" event={"ID":"504278ff-33da-4193-98fe-70bd79e0d35b","Type":"ContainerStarted","Data":"fbd4976ca251067e5725d0d33c0e92f1e46d8da923f7d50005ba0bc06a072a42"}
Jul 13 22:15:03.364361 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:03.364154    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb" event={"ID":"504278ff-33da-4193-98fe-70bd79e0d35b","Type":"ContainerStarted","Data":"ad939decfbde48a45c55d33d944c018a5fea26ec563364956930222c1d7190a9"}
Jul 13 22:15:03.384540 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:03.384467    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb" podStartSLOduration=1.384449644 podStartE2EDuration="1.384449644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:15:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:15:03.383054872 +0000 UTC m=+2745.305103789" watchObservedRunningTime="2026-07-13 22:15:03.384449644 +0000 UTC m=+2745.306498573"
Jul 13 22:15:07.738919 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:07.738874    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml" podUID="dd25585c-450c-4bd7-b8fe-6fc466044baa" containerName="main" containerID="cri-o://1bbe08717d1064bda7af61ef7890d230b0539460edd9aaaced27a05e03ba34a9" gracePeriod=45
Jul 13 22:15:08.002210 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.002135    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:15:08.118853 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.118816    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-home\") pod \"dd25585c-450c-4bd7-b8fe-6fc466044baa\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") "
Jul 13 22:15:08.119062 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.118864    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-model-cache\") pod \"dd25585c-450c-4bd7-b8fe-6fc466044baa\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") "
Jul 13 22:15:08.119062 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.118886    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-dshm\") pod \"dd25585c-450c-4bd7-b8fe-6fc466044baa\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") "
Jul 13 22:15:08.119062 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.118937    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-tmp-dir\") pod \"dd25585c-450c-4bd7-b8fe-6fc466044baa\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") "
Jul 13 22:15:08.119062 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.118994    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2t6j\" (UniqueName: \"kubernetes.io/projected/dd25585c-450c-4bd7-b8fe-6fc466044baa-kube-api-access-n2t6j\") pod \"dd25585c-450c-4bd7-b8fe-6fc466044baa\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") "
Jul 13 22:15:08.119062 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.119024    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd25585c-450c-4bd7-b8fe-6fc466044baa-tls-certs\") pod \"dd25585c-450c-4bd7-b8fe-6fc466044baa\" (UID: \"dd25585c-450c-4bd7-b8fe-6fc466044baa\") "
Jul 13 22:15:08.119337 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.119132    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-home" (OuterVolumeSpecName: "home") pod "dd25585c-450c-4bd7-b8fe-6fc466044baa" (UID: "dd25585c-450c-4bd7-b8fe-6fc466044baa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:15:08.119337 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.119156    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-model-cache" (OuterVolumeSpecName: "model-cache") pod "dd25585c-450c-4bd7-b8fe-6fc466044baa" (UID: "dd25585c-450c-4bd7-b8fe-6fc466044baa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:15:08.119337 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.119251    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dd25585c-450c-4bd7-b8fe-6fc466044baa" (UID: "dd25585c-450c-4bd7-b8fe-6fc466044baa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:15:08.119485 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.119412    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-home\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:15:08.119485 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.119426    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-model-cache\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:15:08.119485 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.119439    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-tmp-dir\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:15:08.121464 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.121435    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd25585c-450c-4bd7-b8fe-6fc466044baa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dd25585c-450c-4bd7-b8fe-6fc466044baa" (UID: "dd25585c-450c-4bd7-b8fe-6fc466044baa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:15:08.121665 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.121475    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd25585c-450c-4bd7-b8fe-6fc466044baa-kube-api-access-n2t6j" (OuterVolumeSpecName: "kube-api-access-n2t6j") pod "dd25585c-450c-4bd7-b8fe-6fc466044baa" (UID: "dd25585c-450c-4bd7-b8fe-6fc466044baa"). InnerVolumeSpecName "kube-api-access-n2t6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:15:08.121665 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.121505    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-dshm" (OuterVolumeSpecName: "dshm") pod "dd25585c-450c-4bd7-b8fe-6fc466044baa" (UID: "dd25585c-450c-4bd7-b8fe-6fc466044baa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:15:08.220509 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.220470    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd25585c-450c-4bd7-b8fe-6fc466044baa-dshm\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:15:08.220509 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.220501    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2t6j\" (UniqueName: \"kubernetes.io/projected/dd25585c-450c-4bd7-b8fe-6fc466044baa-kube-api-access-n2t6j\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:15:08.220509 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.220515    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd25585c-450c-4bd7-b8fe-6fc466044baa-tls-certs\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:15:08.397708 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.397673    2564 generic.go:356] "Generic (PLEG): container finished" podID="dd25585c-450c-4bd7-b8fe-6fc466044baa" containerID="1bbe08717d1064bda7af61ef7890d230b0539460edd9aaaced27a05e03ba34a9" exitCode=0
Jul 13 22:15:08.397708 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.397708    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"
Jul 13 22:15:08.397943 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.397746    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml" event={"ID":"dd25585c-450c-4bd7-b8fe-6fc466044baa","Type":"ContainerDied","Data":"1bbe08717d1064bda7af61ef7890d230b0539460edd9aaaced27a05e03ba34a9"}
Jul 13 22:15:08.397943 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.397795    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml" event={"ID":"dd25585c-450c-4bd7-b8fe-6fc466044baa","Type":"ContainerDied","Data":"54f5ce09bbc65ae98a5418841abd6b866d72b877754637517fab8f7b81a44a40"}
Jul 13 22:15:08.397943 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.397818    2564 scope.go:117] "RemoveContainer" containerID="1bbe08717d1064bda7af61ef7890d230b0539460edd9aaaced27a05e03ba34a9"
Jul 13 22:15:08.407799 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.407773    2564 scope.go:117] "RemoveContainer" containerID="1bbe08717d1064bda7af61ef7890d230b0539460edd9aaaced27a05e03ba34a9"
Jul 13 22:15:08.408106 ip-10-0-140-98 kubenswrapper[2564]: E0713 22:15:08.408086    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bbe08717d1064bda7af61ef7890d230b0539460edd9aaaced27a05e03ba34a9\": container with ID starting with 1bbe08717d1064bda7af61ef7890d230b0539460edd9aaaced27a05e03ba34a9 not found: ID does not exist" containerID="1bbe08717d1064bda7af61ef7890d230b0539460edd9aaaced27a05e03ba34a9"
Jul 13 22:15:08.408169 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.408120    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bbe08717d1064bda7af61ef7890d230b0539460edd9aaaced27a05e03ba34a9"} err="failed to get container status \"1bbe08717d1064bda7af61ef7890d230b0539460edd9aaaced27a05e03ba34a9\": rpc error: code = NotFound desc = could not find container \"1bbe08717d1064bda7af61ef7890d230b0539460edd9aaaced27a05e03ba34a9\": container with ID starting with 1bbe08717d1064bda7af61ef7890d230b0539460edd9aaaced27a05e03ba34a9 not found: ID does not exist"
Jul 13 22:15:08.418834 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.418804    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"]
Jul 13 22:15:08.422495 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.422468    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/auth-test-model-f89343-kserve-76df657967-lstml"]
Jul 13 22:15:08.474731 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:08.474695    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd25585c-450c-4bd7-b8fe-6fc466044baa" path="/var/lib/kubelet/pods/dd25585c-450c-4bd7-b8fe-6fc466044baa/volumes"
Jul 13 22:15:12.946815 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:12.946776    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:12.947284 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:12.947064    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:12.959460 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:12.959436    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:13.438076 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:13.438050    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:15:38.834181 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.834142    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"]
Jul 13 22:15:38.834784 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.834766    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd25585c-450c-4bd7-b8fe-6fc466044baa" containerName="main"
Jul 13 22:15:38.834837 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.834787    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd25585c-450c-4bd7-b8fe-6fc466044baa" containerName="main"
Jul 13 22:15:38.834900 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.834889    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd25585c-450c-4bd7-b8fe-6fc466044baa" containerName="main"
Jul 13 22:15:38.838381 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.838358    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:38.841621 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.841594    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d8932b\"/\"kube-root-ca.crt\""
Jul 13 22:15:38.841792 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.841596    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d8932b\"/\"sub-test-model-d8932b-kserve-self-signed-certs\""
Jul 13 22:15:38.841792 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.841596    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d8932b\"/\"openshift-service-ca.crt\""
Jul 13 22:15:38.841792 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.841708    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d8932b\"/\"default-dockercfg-hdwjt\""
Jul 13 22:15:38.847334 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.847311    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"]
Jul 13 22:15:38.905207 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.905169    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-model-cache\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:38.905207 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.905205    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e522cadc-c9c6-4e75-a85e-25877fa41aa1-tls-certs\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:38.905430 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.905233    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-dshm\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:38.905430 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.905255    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxv58\" (UniqueName: \"kubernetes.io/projected/e522cadc-c9c6-4e75-a85e-25877fa41aa1-kube-api-access-vxv58\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:38.905430 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.905329    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-tmp-dir\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:38.905430 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:38.905419    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-home\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.006043 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.006012    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-tmp-dir\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.006227 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.006089    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-home\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.006227 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.006134    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-model-cache\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.006227 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.006149    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e522cadc-c9c6-4e75-a85e-25877fa41aa1-tls-certs\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.006396 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.006275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-dshm\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.006396 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.006318    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxv58\" (UniqueName: \"kubernetes.io/projected/e522cadc-c9c6-4e75-a85e-25877fa41aa1-kube-api-access-vxv58\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.006505 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.006459    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-tmp-dir\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.006505 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.006488    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-model-cache\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.006613 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.006544    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-home\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.008477 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.008457    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-dshm\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.008773 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.008753    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e522cadc-c9c6-4e75-a85e-25877fa41aa1-tls-certs\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.015485 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.015461    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxv58\" (UniqueName: \"kubernetes.io/projected/e522cadc-c9c6-4e75-a85e-25877fa41aa1-kube-api-access-vxv58\") pod \"sub-test-model-d8932b-kserve-9fb7b948c-pv84c\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") " pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.149614 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.149519    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:39.293403 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.293377    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"]
Jul 13 22:15:39.295815 ip-10-0-140-98 kubenswrapper[2564]: W0713 22:15:39.295787    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode522cadc_c9c6_4e75_a85e_25877fa41aa1.slice/crio-967a9bc08c6e75ff8c9cf8085cad6d204cfdb154989d7369e99d95078691e881 WatchSource:0}: Error finding container 967a9bc08c6e75ff8c9cf8085cad6d204cfdb154989d7369e99d95078691e881: Status 404 returned error can't find the container with id 967a9bc08c6e75ff8c9cf8085cad6d204cfdb154989d7369e99d95078691e881
Jul 13 22:15:39.562852 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.562806    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c" event={"ID":"e522cadc-c9c6-4e75-a85e-25877fa41aa1","Type":"ContainerStarted","Data":"d2f6b02655603c0eede05e49cc625d49156c6bf302c9c12604875df5512a638e"}
Jul 13 22:15:39.563041 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.562858    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c" event={"ID":"e522cadc-c9c6-4e75-a85e-25877fa41aa1","Type":"ContainerStarted","Data":"967a9bc08c6e75ff8c9cf8085cad6d204cfdb154989d7369e99d95078691e881"}
Jul 13 22:15:39.579719 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:39.579663    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c" podStartSLOduration=1.579628782 podStartE2EDuration="1.579628782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:15:39.578536795 +0000 UTC m=+2781.500585705" watchObservedRunningTime="2026-07-13 22:15:39.579628782 +0000 UTC m=+2781.501677699"
Jul 13 22:15:49.150057 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:49.149964    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:49.150057 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:49.150032    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:49.162749 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:49.162716    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:15:49.625157 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:15:49.625128    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:16:25.010875 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:25.010829    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"]
Jul 13 22:16:25.908029 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:25.907990    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"]
Jul 13 22:16:40.031786 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.031716    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb" podUID="504278ff-33da-4193-98fe-70bd79e0d35b" containerName="main" containerID="cri-o://fbd4976ca251067e5725d0d33c0e92f1e46d8da923f7d50005ba0bc06a072a42" gracePeriod=45
Jul 13 22:16:40.293437 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.293368    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:16:40.367155 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.367119    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-home\") pod \"504278ff-33da-4193-98fe-70bd79e0d35b\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") "
Jul 13 22:16:40.367353 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.367181    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-tmp-dir\") pod \"504278ff-33da-4193-98fe-70bd79e0d35b\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") "
Jul 13 22:16:40.367353 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.367268    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkw64\" (UniqueName: \"kubernetes.io/projected/504278ff-33da-4193-98fe-70bd79e0d35b-kube-api-access-dkw64\") pod \"504278ff-33da-4193-98fe-70bd79e0d35b\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") "
Jul 13 22:16:40.367353 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.367297    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/504278ff-33da-4193-98fe-70bd79e0d35b-tls-certs\") pod \"504278ff-33da-4193-98fe-70bd79e0d35b\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") "
Jul 13 22:16:40.367518 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.367386    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-model-cache\") pod \"504278ff-33da-4193-98fe-70bd79e0d35b\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") "
Jul 13 22:16:40.367518 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.367414    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-dshm\") pod \"504278ff-33da-4193-98fe-70bd79e0d35b\" (UID: \"504278ff-33da-4193-98fe-70bd79e0d35b\") "
Jul 13 22:16:40.367518 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.367445    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-home" (OuterVolumeSpecName: "home") pod "504278ff-33da-4193-98fe-70bd79e0d35b" (UID: "504278ff-33da-4193-98fe-70bd79e0d35b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:16:40.367518 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.367500    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "504278ff-33da-4193-98fe-70bd79e0d35b" (UID: "504278ff-33da-4193-98fe-70bd79e0d35b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:16:40.367780 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.367728    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-home\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:16:40.367780 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.367752    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-tmp-dir\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:16:40.367780 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.367752    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-model-cache" (OuterVolumeSpecName: "model-cache") pod "504278ff-33da-4193-98fe-70bd79e0d35b" (UID: "504278ff-33da-4193-98fe-70bd79e0d35b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:16:40.369641 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.369612    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-dshm" (OuterVolumeSpecName: "dshm") pod "504278ff-33da-4193-98fe-70bd79e0d35b" (UID: "504278ff-33da-4193-98fe-70bd79e0d35b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:16:40.369796 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.369710    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/504278ff-33da-4193-98fe-70bd79e0d35b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "504278ff-33da-4193-98fe-70bd79e0d35b" (UID: "504278ff-33da-4193-98fe-70bd79e0d35b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:16:40.369796 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.369757    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/504278ff-33da-4193-98fe-70bd79e0d35b-kube-api-access-dkw64" (OuterVolumeSpecName: "kube-api-access-dkw64") pod "504278ff-33da-4193-98fe-70bd79e0d35b" (UID: "504278ff-33da-4193-98fe-70bd79e0d35b"). InnerVolumeSpecName "kube-api-access-dkw64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:16:40.468435 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.468396    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkw64\" (UniqueName: \"kubernetes.io/projected/504278ff-33da-4193-98fe-70bd79e0d35b-kube-api-access-dkw64\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:16:40.468435 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.468429    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/504278ff-33da-4193-98fe-70bd79e0d35b-tls-certs\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:16:40.468435 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.468441    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-model-cache\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:16:40.468775 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.468449    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/504278ff-33da-4193-98fe-70bd79e0d35b-dshm\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:16:40.812609 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.812574    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"]
Jul 13 22:16:40.812994 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.812980    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504278ff-33da-4193-98fe-70bd79e0d35b" containerName="main"
Jul 13 22:16:40.813047 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.812996    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="504278ff-33da-4193-98fe-70bd79e0d35b" containerName="main"
Jul 13 22:16:40.813083 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.813076    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="504278ff-33da-4193-98fe-70bd79e0d35b" containerName="main"
Jul 13 22:16:40.816206 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.816187    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.818438 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.818414    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f89343\"/\"rate-test-model-f89343-kserve-self-signed-certs\""
Jul 13 22:16:40.826214 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.826192    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"]
Jul 13 22:16:40.871729 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.871696    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th2jr\" (UniqueName: \"kubernetes.io/projected/873f2d3c-0e91-4087-94eb-eea64c9d354e-kube-api-access-th2jr\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.871918 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.871743    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-dshm\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.871918 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.871852    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-tmp-dir\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.871918 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.871879    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/873f2d3c-0e91-4087-94eb-eea64c9d354e-tls-certs\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.872044 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.871914    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-model-cache\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.872044 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.871944    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-home\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.876114 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.876089    2564 generic.go:356] "Generic (PLEG): container finished" podID="504278ff-33da-4193-98fe-70bd79e0d35b" containerID="fbd4976ca251067e5725d0d33c0e92f1e46d8da923f7d50005ba0bc06a072a42" exitCode=0
Jul 13 22:16:40.876249 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.876134    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"
Jul 13 22:16:40.876249 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.876175    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb" event={"ID":"504278ff-33da-4193-98fe-70bd79e0d35b","Type":"ContainerDied","Data":"fbd4976ca251067e5725d0d33c0e92f1e46d8da923f7d50005ba0bc06a072a42"}
Jul 13 22:16:40.876249 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.876218    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb" event={"ID":"504278ff-33da-4193-98fe-70bd79e0d35b","Type":"ContainerDied","Data":"ad939decfbde48a45c55d33d944c018a5fea26ec563364956930222c1d7190a9"}
Jul 13 22:16:40.876249 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.876241    2564 scope.go:117] "RemoveContainer" containerID="fbd4976ca251067e5725d0d33c0e92f1e46d8da923f7d50005ba0bc06a072a42"
Jul 13 22:16:40.885403 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.885380    2564 scope.go:117] "RemoveContainer" containerID="fbd4976ca251067e5725d0d33c0e92f1e46d8da923f7d50005ba0bc06a072a42"
Jul 13 22:16:40.885674 ip-10-0-140-98 kubenswrapper[2564]: E0713 22:16:40.885639    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbd4976ca251067e5725d0d33c0e92f1e46d8da923f7d50005ba0bc06a072a42\": container with ID starting with fbd4976ca251067e5725d0d33c0e92f1e46d8da923f7d50005ba0bc06a072a42 not found: ID does not exist" containerID="fbd4976ca251067e5725d0d33c0e92f1e46d8da923f7d50005ba0bc06a072a42"
Jul 13 22:16:40.885816 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.885686    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbd4976ca251067e5725d0d33c0e92f1e46d8da923f7d50005ba0bc06a072a42"} err="failed to get container status \"fbd4976ca251067e5725d0d33c0e92f1e46d8da923f7d50005ba0bc06a072a42\": rpc error: code = NotFound desc = could not find container \"fbd4976ca251067e5725d0d33c0e92f1e46d8da923f7d50005ba0bc06a072a42\": container with ID starting with fbd4976ca251067e5725d0d33c0e92f1e46d8da923f7d50005ba0bc06a072a42 not found: ID does not exist"
Jul 13 22:16:40.896099 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.896066    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"]
Jul 13 22:16:40.900266 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.900237    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/sub-test-model-f89343-kserve-6f46cf768f-lv4zb"]
Jul 13 22:16:40.931436 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.931361    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c" podUID="e522cadc-c9c6-4e75-a85e-25877fa41aa1" containerName="main" containerID="cri-o://d2f6b02655603c0eede05e49cc625d49156c6bf302c9c12604875df5512a638e" gracePeriod=45
Jul 13 22:16:40.973315 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.973283    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-tmp-dir\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.973315 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.973316    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/873f2d3c-0e91-4087-94eb-eea64c9d354e-tls-certs\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.973497 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.973351    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-model-cache\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.973497 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.973464    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-home\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.973581 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.973513    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th2jr\" (UniqueName: \"kubernetes.io/projected/873f2d3c-0e91-4087-94eb-eea64c9d354e-kube-api-access-th2jr\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.973581 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.973572    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-dshm\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.973749 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.973723    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-model-cache\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.973842 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.973813    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-tmp-dir\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.973909 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.973865    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-home\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.975689 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.975671    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-dshm\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.975895 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.975878    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/873f2d3c-0e91-4087-94eb-eea64c9d354e-tls-certs\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:40.981836 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:40.981817    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th2jr\" (UniqueName: \"kubernetes.io/projected/873f2d3c-0e91-4087-94eb-eea64c9d354e-kube-api-access-th2jr\") pod \"rate-test-model-f89343-kserve-699db75b4c-28t9g\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") " pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:41.128062 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.128022    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:41.201785 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.200302    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:16:41.276353 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.276310    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-dshm\") pod \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") "
Jul 13 22:16:41.276353 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.276352    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-home\") pod \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") "
Jul 13 22:16:41.276611 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.276420    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-tmp-dir\") pod \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") "
Jul 13 22:16:41.276930 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.276895    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-model-cache\") pod \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") "
Jul 13 22:16:41.277076 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.276940    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e522cadc-c9c6-4e75-a85e-25877fa41aa1-tls-certs\") pod \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") "
Jul 13 22:16:41.277076 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.276966    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxv58\" (UniqueName: \"kubernetes.io/projected/e522cadc-c9c6-4e75-a85e-25877fa41aa1-kube-api-access-vxv58\") pod \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\" (UID: \"e522cadc-c9c6-4e75-a85e-25877fa41aa1\") "
Jul 13 22:16:41.277428 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.277252    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-model-cache" (OuterVolumeSpecName: "model-cache") pod "e522cadc-c9c6-4e75-a85e-25877fa41aa1" (UID: "e522cadc-c9c6-4e75-a85e-25877fa41aa1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:16:41.277428 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.277407    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-model-cache\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:16:41.277597 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.277427    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e522cadc-c9c6-4e75-a85e-25877fa41aa1" (UID: "e522cadc-c9c6-4e75-a85e-25877fa41aa1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:16:41.277597 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.277491    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-home" (OuterVolumeSpecName: "home") pod "e522cadc-c9c6-4e75-a85e-25877fa41aa1" (UID: "e522cadc-c9c6-4e75-a85e-25877fa41aa1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:16:41.279180 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.279155    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-dshm" (OuterVolumeSpecName: "dshm") pod "e522cadc-c9c6-4e75-a85e-25877fa41aa1" (UID: "e522cadc-c9c6-4e75-a85e-25877fa41aa1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:16:41.279297 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.279201    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"]
Jul 13 22:16:41.279631 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.279609    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e522cadc-c9c6-4e75-a85e-25877fa41aa1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e522cadc-c9c6-4e75-a85e-25877fa41aa1" (UID: "e522cadc-c9c6-4e75-a85e-25877fa41aa1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:16:41.279711 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.279625    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e522cadc-c9c6-4e75-a85e-25877fa41aa1-kube-api-access-vxv58" (OuterVolumeSpecName: "kube-api-access-vxv58") pod "e522cadc-c9c6-4e75-a85e-25877fa41aa1" (UID: "e522cadc-c9c6-4e75-a85e-25877fa41aa1"). InnerVolumeSpecName "kube-api-access-vxv58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:16:41.281785 ip-10-0-140-98 kubenswrapper[2564]: W0713 22:16:41.281765    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod873f2d3c_0e91_4087_94eb_eea64c9d354e.slice/crio-737fe915a61d68bc69d0403f14ebb19d06f9ccfd2f7015ef9b5ba4ffd03e8b8e WatchSource:0}: Error finding container 737fe915a61d68bc69d0403f14ebb19d06f9ccfd2f7015ef9b5ba4ffd03e8b8e: Status 404 returned error can't find the container with id 737fe915a61d68bc69d0403f14ebb19d06f9ccfd2f7015ef9b5ba4ffd03e8b8e
Jul 13 22:16:41.378473 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.378440    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e522cadc-c9c6-4e75-a85e-25877fa41aa1-tls-certs\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:16:41.378473 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.378471    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxv58\" (UniqueName: \"kubernetes.io/projected/e522cadc-c9c6-4e75-a85e-25877fa41aa1-kube-api-access-vxv58\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:16:41.378473 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.378480    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-dshm\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:16:41.378738 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.378488    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-home\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:16:41.378738 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.378497    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e522cadc-c9c6-4e75-a85e-25877fa41aa1-tmp-dir\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:16:41.883294 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.883253    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g" event={"ID":"873f2d3c-0e91-4087-94eb-eea64c9d354e","Type":"ContainerStarted","Data":"e4dbab16e124c83c009c0c56bbcc3fcea5e7ce26af49bbe1cb2f498b142ba381"}
Jul 13 22:16:41.883510 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.883306    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g" event={"ID":"873f2d3c-0e91-4087-94eb-eea64c9d354e","Type":"ContainerStarted","Data":"737fe915a61d68bc69d0403f14ebb19d06f9ccfd2f7015ef9b5ba4ffd03e8b8e"}
Jul 13 22:16:41.884600 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.884571    2564 generic.go:356] "Generic (PLEG): container finished" podID="e522cadc-c9c6-4e75-a85e-25877fa41aa1" containerID="d2f6b02655603c0eede05e49cc625d49156c6bf302c9c12604875df5512a638e" exitCode=0
Jul 13 22:16:41.884712 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.884609    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"
Jul 13 22:16:41.884712 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.884671    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c" event={"ID":"e522cadc-c9c6-4e75-a85e-25877fa41aa1","Type":"ContainerDied","Data":"d2f6b02655603c0eede05e49cc625d49156c6bf302c9c12604875df5512a638e"}
Jul 13 22:16:41.884712 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.884704    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c" event={"ID":"e522cadc-c9c6-4e75-a85e-25877fa41aa1","Type":"ContainerDied","Data":"967a9bc08c6e75ff8c9cf8085cad6d204cfdb154989d7369e99d95078691e881"}
Jul 13 22:16:41.884841 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.884720    2564 scope.go:117] "RemoveContainer" containerID="d2f6b02655603c0eede05e49cc625d49156c6bf302c9c12604875df5512a638e"
Jul 13 22:16:41.894175 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.894154    2564 scope.go:117] "RemoveContainer" containerID="d2f6b02655603c0eede05e49cc625d49156c6bf302c9c12604875df5512a638e"
Jul 13 22:16:41.894428 ip-10-0-140-98 kubenswrapper[2564]: E0713 22:16:41.894398    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2f6b02655603c0eede05e49cc625d49156c6bf302c9c12604875df5512a638e\": container with ID starting with d2f6b02655603c0eede05e49cc625d49156c6bf302c9c12604875df5512a638e not found: ID does not exist" containerID="d2f6b02655603c0eede05e49cc625d49156c6bf302c9c12604875df5512a638e"
Jul 13 22:16:41.894493 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.894436    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2f6b02655603c0eede05e49cc625d49156c6bf302c9c12604875df5512a638e"} err="failed to get container status \"d2f6b02655603c0eede05e49cc625d49156c6bf302c9c12604875df5512a638e\": rpc error: code = NotFound desc = could not find container \"d2f6b02655603c0eede05e49cc625d49156c6bf302c9c12604875df5512a638e\": container with ID starting with d2f6b02655603c0eede05e49cc625d49156c6bf302c9c12604875df5512a638e not found: ID does not exist"
Jul 13 22:16:41.900408 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.900363    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g" podStartSLOduration=1.900346781 podStartE2EDuration="1.900346781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 22:16:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:16:41.898130216 +0000 UTC m=+2843.820179133" watchObservedRunningTime="2026-07-13 22:16:41.900346781 +0000 UTC m=+2843.822395697"
Jul 13 22:16:41.911355 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.911326    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"]
Jul 13 22:16:41.915119 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:41.915094    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d8932b/sub-test-model-d8932b-kserve-9fb7b948c-pv84c"]
Jul 13 22:16:42.473834 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:42.473798    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="504278ff-33da-4193-98fe-70bd79e0d35b" path="/var/lib/kubelet/pods/504278ff-33da-4193-98fe-70bd79e0d35b/volumes"
Jul 13 22:16:42.474249 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:42.474211    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e522cadc-c9c6-4e75-a85e-25877fa41aa1" path="/var/lib/kubelet/pods/e522cadc-c9c6-4e75-a85e-25877fa41aa1/volumes"
Jul 13 22:16:51.128987 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:51.128944    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:51.129476 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:51.129005    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:51.141596 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:51.141566    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:16:51.950858 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:16:51.950825    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:17:54.909775 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:54.909742    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"]
Jul 13 22:17:56.904763 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:56.904730    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7605-139927599906096"]
Jul 13 22:17:56.905144 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:56.905135    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e522cadc-c9c6-4e75-a85e-25877fa41aa1" containerName="main"
Jul 13 22:17:56.905192 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:56.905147    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e522cadc-c9c6-4e75-a85e-25877fa41aa1" containerName="main"
Jul 13 22:17:56.905228 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:56.905209    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e522cadc-c9c6-4e75-a85e-25877fa41aa1" containerName="main"
Jul 13 22:17:56.908345 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:56.908315    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927599906096"
Jul 13 22:17:56.910703 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:56.910684    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6g8h9\""
Jul 13 22:17:56.913474 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:56.913453    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-139927599906096"]
Jul 13 22:17:57.087373 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:57.087338    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdtr5\" (UniqueName: \"kubernetes.io/projected/523a3fe4-8f36-4f66-bcfc-e1216c382f61-kube-api-access-mdtr5\") pod \"test-curl-7605-139927599906096\" (UID: \"523a3fe4-8f36-4f66-bcfc-e1216c382f61\") " pod="opendatahub/test-curl-7605-139927599906096"
Jul 13 22:17:57.188573 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:57.188472    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdtr5\" (UniqueName: \"kubernetes.io/projected/523a3fe4-8f36-4f66-bcfc-e1216c382f61-kube-api-access-mdtr5\") pod \"test-curl-7605-139927599906096\" (UID: \"523a3fe4-8f36-4f66-bcfc-e1216c382f61\") " pod="opendatahub/test-curl-7605-139927599906096"
Jul 13 22:17:57.196451 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:57.196418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdtr5\" (UniqueName: \"kubernetes.io/projected/523a3fe4-8f36-4f66-bcfc-e1216c382f61-kube-api-access-mdtr5\") pod \"test-curl-7605-139927599906096\" (UID: \"523a3fe4-8f36-4f66-bcfc-e1216c382f61\") " pod="opendatahub/test-curl-7605-139927599906096"
Jul 13 22:17:57.219296 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:57.219264    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927599906096"
Jul 13 22:17:57.352605 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:57.352501    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-139927599906096"]
Jul 13 22:17:57.354857 ip-10-0-140-98 kubenswrapper[2564]: W0713 22:17:57.354832    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod523a3fe4_8f36_4f66_bcfc_e1216c382f61.slice/crio-5f043b2045e23ec965c5dc5662a63c7942a3b7e0e8ed5713132bb837189ab9dc WatchSource:0}: Error finding container 5f043b2045e23ec965c5dc5662a63c7942a3b7e0e8ed5713132bb837189ab9dc: Status 404 returned error can't find the container with id 5f043b2045e23ec965c5dc5662a63c7942a3b7e0e8ed5713132bb837189ab9dc
Jul 13 22:17:57.356063 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:57.356048    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 22:17:58.266709 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:58.266680    2564 generic.go:356] "Generic (PLEG): container finished" podID="523a3fe4-8f36-4f66-bcfc-e1216c382f61" containerID="7ada0f0e049d2aec398ae28ced7101e022dcc055e63dce540e7fc044d95fa323" exitCode=0
Jul 13 22:17:58.267078 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:58.266773    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-139927599906096" event={"ID":"523a3fe4-8f36-4f66-bcfc-e1216c382f61","Type":"ContainerDied","Data":"7ada0f0e049d2aec398ae28ced7101e022dcc055e63dce540e7fc044d95fa323"}
Jul 13 22:17:58.267078 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:58.266822    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-139927599906096" event={"ID":"523a3fe4-8f36-4f66-bcfc-e1216c382f61","Type":"ContainerStarted","Data":"5f043b2045e23ec965c5dc5662a63c7942a3b7e0e8ed5713132bb837189ab9dc"}
Jul 13 22:17:59.415104 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.415076    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927599906096"
Jul 13 22:17:59.452128 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.452104    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7605-139927599906096_523a3fe4-8f36-4f66-bcfc-e1216c382f61/test-curl-7605-139927599906096/0.log"
Jul 13 22:17:59.475081 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.475046    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7605-139927599906096"]
Jul 13 22:17:59.478413 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.478378    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7605-139927599906096"]
Jul 13 22:17:59.605110 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.605020    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7605-139927600891056"]
Jul 13 22:17:59.605531 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.605466    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="523a3fe4-8f36-4f66-bcfc-e1216c382f61" containerName="test-curl-7605-139927599906096"
Jul 13 22:17:59.605531 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.605486    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="523a3fe4-8f36-4f66-bcfc-e1216c382f61" containerName="test-curl-7605-139927599906096"
Jul 13 22:17:59.605728 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.605603    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="523a3fe4-8f36-4f66-bcfc-e1216c382f61" containerName="test-curl-7605-139927599906096"
Jul 13 22:17:59.607267 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.607238    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdtr5\" (UniqueName: \"kubernetes.io/projected/523a3fe4-8f36-4f66-bcfc-e1216c382f61-kube-api-access-mdtr5\") pod \"523a3fe4-8f36-4f66-bcfc-e1216c382f61\" (UID: \"523a3fe4-8f36-4f66-bcfc-e1216c382f61\") "
Jul 13 22:17:59.608815 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.608796    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600891056"
Jul 13 22:17:59.609537 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.609515    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/523a3fe4-8f36-4f66-bcfc-e1216c382f61-kube-api-access-mdtr5" (OuterVolumeSpecName: "kube-api-access-mdtr5") pod "523a3fe4-8f36-4f66-bcfc-e1216c382f61" (UID: "523a3fe4-8f36-4f66-bcfc-e1216c382f61"). InnerVolumeSpecName "kube-api-access-mdtr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:17:59.613861 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.613835    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-139927600891056"]
Jul 13 22:17:59.708321 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.708279    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwxks\" (UniqueName: \"kubernetes.io/projected/36f513ce-e463-4af7-abdb-9b0141d38b59-kube-api-access-zwxks\") pod \"test-curl-7605-139927600891056\" (UID: \"36f513ce-e463-4af7-abdb-9b0141d38b59\") " pod="opendatahub/test-curl-7605-139927600891056"
Jul 13 22:17:59.708494 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.708435    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdtr5\" (UniqueName: \"kubernetes.io/projected/523a3fe4-8f36-4f66-bcfc-e1216c382f61-kube-api-access-mdtr5\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:17:59.809133 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.809092    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwxks\" (UniqueName: \"kubernetes.io/projected/36f513ce-e463-4af7-abdb-9b0141d38b59-kube-api-access-zwxks\") pod \"test-curl-7605-139927600891056\" (UID: \"36f513ce-e463-4af7-abdb-9b0141d38b59\") " pod="opendatahub/test-curl-7605-139927600891056"
Jul 13 22:17:59.816831 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.816789    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwxks\" (UniqueName: \"kubernetes.io/projected/36f513ce-e463-4af7-abdb-9b0141d38b59-kube-api-access-zwxks\") pod \"test-curl-7605-139927600891056\" (UID: \"36f513ce-e463-4af7-abdb-9b0141d38b59\") " pod="opendatahub/test-curl-7605-139927600891056"
Jul 13 22:17:59.927523 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:17:59.927430    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600891056"
Jul 13 22:18:00.064853 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:00.064826    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-139927600891056"]
Jul 13 22:18:00.067244 ip-10-0-140-98 kubenswrapper[2564]: W0713 22:18:00.067215    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36f513ce_e463_4af7_abdb_9b0141d38b59.slice/crio-dff06f88af029e5ea77f776eeba1a1bf68abe808671014848cfc895183bc70af WatchSource:0}: Error finding container dff06f88af029e5ea77f776eeba1a1bf68abe808671014848cfc895183bc70af: Status 404 returned error can't find the container with id dff06f88af029e5ea77f776eeba1a1bf68abe808671014848cfc895183bc70af
Jul 13 22:18:00.277884 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:00.277849    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-139927600891056" event={"ID":"36f513ce-e463-4af7-abdb-9b0141d38b59","Type":"ContainerStarted","Data":"dff06f88af029e5ea77f776eeba1a1bf68abe808671014848cfc895183bc70af"}
Jul 13 22:18:00.279129 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:00.279109    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f043b2045e23ec965c5dc5662a63c7942a3b7e0e8ed5713132bb837189ab9dc"
Jul 13 22:18:00.279129 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:00.279124    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927599906096"
Jul 13 22:18:00.289792 ip-10-0-140-98 kubenswrapper[2564]: E0713 22:18:00.289759    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7605-139927599906096\" is forbidden: User \"system:node:ip-10-0-140-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-98.ec2.internal' and this object" podUID="523a3fe4-8f36-4f66-bcfc-e1216c382f61" pod="opendatahub/test-curl-7605-139927599906096"
Jul 13 22:18:00.478737 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:00.478704    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="523a3fe4-8f36-4f66-bcfc-e1216c382f61" path="/var/lib/kubelet/pods/523a3fe4-8f36-4f66-bcfc-e1216c382f61/volumes"
Jul 13 22:18:01.285799 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:01.285763    2564 generic.go:356] "Generic (PLEG): container finished" podID="36f513ce-e463-4af7-abdb-9b0141d38b59" containerID="599371babf775367ee323f35d2a1273102452fd5c307494b8291fb7a113af29a" exitCode=0
Jul 13 22:18:01.285974 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:01.285858    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-139927600891056" event={"ID":"36f513ce-e463-4af7-abdb-9b0141d38b59","Type":"ContainerDied","Data":"599371babf775367ee323f35d2a1273102452fd5c307494b8291fb7a113af29a"}
Jul 13 22:18:02.431954 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:02.431931    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600891056"
Jul 13 22:18:02.450813 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:02.450787    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7605-139927600891056_36f513ce-e463-4af7-abdb-9b0141d38b59/test-curl-7605-139927600891056/0.log"
Jul 13 22:18:02.476005 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:02.475982    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7605-139927600891056"]
Jul 13 22:18:02.476167 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:02.476018    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7605-139927600891056"]
Jul 13 22:18:02.531359 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:02.531321    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwxks\" (UniqueName: \"kubernetes.io/projected/36f513ce-e463-4af7-abdb-9b0141d38b59-kube-api-access-zwxks\") pod \"36f513ce-e463-4af7-abdb-9b0141d38b59\" (UID: \"36f513ce-e463-4af7-abdb-9b0141d38b59\") "
Jul 13 22:18:02.533390 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:02.533360    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36f513ce-e463-4af7-abdb-9b0141d38b59-kube-api-access-zwxks" (OuterVolumeSpecName: "kube-api-access-zwxks") pod "36f513ce-e463-4af7-abdb-9b0141d38b59" (UID: "36f513ce-e463-4af7-abdb-9b0141d38b59"). InnerVolumeSpecName "kube-api-access-zwxks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:18:02.632778 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:02.632696    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwxks\" (UniqueName: \"kubernetes.io/projected/36f513ce-e463-4af7-abdb-9b0141d38b59-kube-api-access-zwxks\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:18:03.297015 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:03.296987    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600891056"
Jul 13 22:18:03.297193 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:03.297033    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dff06f88af029e5ea77f776eeba1a1bf68abe808671014848cfc895183bc70af"
Jul 13 22:18:03.306793 ip-10-0-140-98 kubenswrapper[2564]: E0713 22:18:03.306762    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7605-139927600891056\" is forbidden: User \"system:node:ip-10-0-140-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-98.ec2.internal' and this object" podUID="36f513ce-e463-4af7-abdb-9b0141d38b59" pod="opendatahub/test-curl-7605-139927600891056"
Jul 13 22:18:04.474608 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:04.474566    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36f513ce-e463-4af7-abdb-9b0141d38b59" path="/var/lib/kubelet/pods/36f513ce-e463-4af7-abdb-9b0141d38b59/volumes"
Jul 13 22:18:08.619033 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:08.619000    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7605-139927600976928"]
Jul 13 22:18:08.619416 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:08.619373    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36f513ce-e463-4af7-abdb-9b0141d38b59" containerName="test-curl-7605-139927600891056"
Jul 13 22:18:08.619416 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:08.619383    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="36f513ce-e463-4af7-abdb-9b0141d38b59" containerName="test-curl-7605-139927600891056"
Jul 13 22:18:08.619494 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:08.619460    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="36f513ce-e463-4af7-abdb-9b0141d38b59" containerName="test-curl-7605-139927600891056"
Jul 13 22:18:08.622685 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:08.622668    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:08.624916 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:08.624892    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6g8h9\""
Jul 13 22:18:08.629073 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:08.629047    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-139927600976928"]
Jul 13 22:18:08.688631 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:08.688592    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fms2\" (UniqueName: \"kubernetes.io/projected/05c61939-b5d0-494c-835e-dbf4e9ce0199-kube-api-access-8fms2\") pod \"test-curl-7605-139927600976928\" (UID: \"05c61939-b5d0-494c-835e-dbf4e9ce0199\") " pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:08.789576 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:08.789534    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fms2\" (UniqueName: \"kubernetes.io/projected/05c61939-b5d0-494c-835e-dbf4e9ce0199-kube-api-access-8fms2\") pod \"test-curl-7605-139927600976928\" (UID: \"05c61939-b5d0-494c-835e-dbf4e9ce0199\") " pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:08.797378 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:08.797351    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fms2\" (UniqueName: \"kubernetes.io/projected/05c61939-b5d0-494c-835e-dbf4e9ce0199-kube-api-access-8fms2\") pod \"test-curl-7605-139927600976928\" (UID: \"05c61939-b5d0-494c-835e-dbf4e9ce0199\") " pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:08.933188 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:08.933097    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:09.274474 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:09.274451    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-139927600976928"]
Jul 13 22:18:09.277096 ip-10-0-140-98 kubenswrapper[2564]: W0713 22:18:09.277065    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05c61939_b5d0_494c_835e_dbf4e9ce0199.slice/crio-9e1fd5d3baa21ab9e63c468a78330bfacde5ada5fdc1fb1186aae68ecf13b20d WatchSource:0}: Error finding container 9e1fd5d3baa21ab9e63c468a78330bfacde5ada5fdc1fb1186aae68ecf13b20d: Status 404 returned error can't find the container with id 9e1fd5d3baa21ab9e63c468a78330bfacde5ada5fdc1fb1186aae68ecf13b20d
Jul 13 22:18:09.330055 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:09.330027    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-139927600976928" event={"ID":"05c61939-b5d0-494c-835e-dbf4e9ce0199","Type":"ContainerStarted","Data":"9e1fd5d3baa21ab9e63c468a78330bfacde5ada5fdc1fb1186aae68ecf13b20d"}
Jul 13 22:18:09.930212 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:09.930124    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g" podUID="873f2d3c-0e91-4087-94eb-eea64c9d354e" containerName="main" containerID="cri-o://e4dbab16e124c83c009c0c56bbcc3fcea5e7ce26af49bbe1cb2f498b142ba381" gracePeriod=45
Jul 13 22:18:10.187997 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.187927    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:18:10.304321 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.304282    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-model-cache\") pod \"873f2d3c-0e91-4087-94eb-eea64c9d354e\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") "
Jul 13 22:18:10.304529 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.304350    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-dshm\") pod \"873f2d3c-0e91-4087-94eb-eea64c9d354e\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") "
Jul 13 22:18:10.304529 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.304386    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-tmp-dir\") pod \"873f2d3c-0e91-4087-94eb-eea64c9d354e\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") "
Jul 13 22:18:10.304529 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.304441    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/873f2d3c-0e91-4087-94eb-eea64c9d354e-tls-certs\") pod \"873f2d3c-0e91-4087-94eb-eea64c9d354e\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") "
Jul 13 22:18:10.304529 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.304483    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-th2jr\" (UniqueName: \"kubernetes.io/projected/873f2d3c-0e91-4087-94eb-eea64c9d354e-kube-api-access-th2jr\") pod \"873f2d3c-0e91-4087-94eb-eea64c9d354e\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") "
Jul 13 22:18:10.304529 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.304509    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-home\") pod \"873f2d3c-0e91-4087-94eb-eea64c9d354e\" (UID: \"873f2d3c-0e91-4087-94eb-eea64c9d354e\") "
Jul 13 22:18:10.304804 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.304563    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-model-cache" (OuterVolumeSpecName: "model-cache") pod "873f2d3c-0e91-4087-94eb-eea64c9d354e" (UID: "873f2d3c-0e91-4087-94eb-eea64c9d354e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:18:10.304804 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.304730    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "873f2d3c-0e91-4087-94eb-eea64c9d354e" (UID: "873f2d3c-0e91-4087-94eb-eea64c9d354e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:18:10.304914 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.304808    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-home" (OuterVolumeSpecName: "home") pod "873f2d3c-0e91-4087-94eb-eea64c9d354e" (UID: "873f2d3c-0e91-4087-94eb-eea64c9d354e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:18:10.304914 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.304875    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-model-cache\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:18:10.304914 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.304888    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-tmp-dir\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:18:10.306726 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.306691    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/873f2d3c-0e91-4087-94eb-eea64c9d354e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "873f2d3c-0e91-4087-94eb-eea64c9d354e" (UID: "873f2d3c-0e91-4087-94eb-eea64c9d354e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 22:18:10.306828 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.306728    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-dshm" (OuterVolumeSpecName: "dshm") pod "873f2d3c-0e91-4087-94eb-eea64c9d354e" (UID: "873f2d3c-0e91-4087-94eb-eea64c9d354e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 22:18:10.306828 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.306731    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/873f2d3c-0e91-4087-94eb-eea64c9d354e-kube-api-access-th2jr" (OuterVolumeSpecName: "kube-api-access-th2jr") pod "873f2d3c-0e91-4087-94eb-eea64c9d354e" (UID: "873f2d3c-0e91-4087-94eb-eea64c9d354e"). InnerVolumeSpecName "kube-api-access-th2jr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:18:10.337769 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.337735    2564 generic.go:356] "Generic (PLEG): container finished" podID="05c61939-b5d0-494c-835e-dbf4e9ce0199" containerID="f31509ba3e71237760a78466b70ce983bc0ff0af79bd9a2c4f9f0cf8a2f9988b" exitCode=0
Jul 13 22:18:10.337935 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.337815    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-139927600976928" event={"ID":"05c61939-b5d0-494c-835e-dbf4e9ce0199","Type":"ContainerDied","Data":"f31509ba3e71237760a78466b70ce983bc0ff0af79bd9a2c4f9f0cf8a2f9988b"}
Jul 13 22:18:10.339186 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.339167    2564 generic.go:356] "Generic (PLEG): container finished" podID="873f2d3c-0e91-4087-94eb-eea64c9d354e" containerID="e4dbab16e124c83c009c0c56bbcc3fcea5e7ce26af49bbe1cb2f498b142ba381" exitCode=0
Jul 13 22:18:10.339265 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.339201    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"
Jul 13 22:18:10.339265 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.339227    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g" event={"ID":"873f2d3c-0e91-4087-94eb-eea64c9d354e","Type":"ContainerDied","Data":"e4dbab16e124c83c009c0c56bbcc3fcea5e7ce26af49bbe1cb2f498b142ba381"}
Jul 13 22:18:10.339265 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.339258    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g" event={"ID":"873f2d3c-0e91-4087-94eb-eea64c9d354e","Type":"ContainerDied","Data":"737fe915a61d68bc69d0403f14ebb19d06f9ccfd2f7015ef9b5ba4ffd03e8b8e"}
Jul 13 22:18:10.339372 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.339276    2564 scope.go:117] "RemoveContainer" containerID="e4dbab16e124c83c009c0c56bbcc3fcea5e7ce26af49bbe1cb2f498b142ba381"
Jul 13 22:18:10.348681 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.348635    2564 scope.go:117] "RemoveContainer" containerID="e4dbab16e124c83c009c0c56bbcc3fcea5e7ce26af49bbe1cb2f498b142ba381"
Jul 13 22:18:10.348967 ip-10-0-140-98 kubenswrapper[2564]: E0713 22:18:10.348946    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4dbab16e124c83c009c0c56bbcc3fcea5e7ce26af49bbe1cb2f498b142ba381\": container with ID starting with e4dbab16e124c83c009c0c56bbcc3fcea5e7ce26af49bbe1cb2f498b142ba381 not found: ID does not exist" containerID="e4dbab16e124c83c009c0c56bbcc3fcea5e7ce26af49bbe1cb2f498b142ba381"
Jul 13 22:18:10.349053 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.348972    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4dbab16e124c83c009c0c56bbcc3fcea5e7ce26af49bbe1cb2f498b142ba381"} err="failed to get container status \"e4dbab16e124c83c009c0c56bbcc3fcea5e7ce26af49bbe1cb2f498b142ba381\": rpc error: code = NotFound desc = could not find container \"e4dbab16e124c83c009c0c56bbcc3fcea5e7ce26af49bbe1cb2f498b142ba381\": container with ID starting with e4dbab16e124c83c009c0c56bbcc3fcea5e7ce26af49bbe1cb2f498b142ba381 not found: ID does not exist"
Jul 13 22:18:10.363467 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.363435    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"]
Jul 13 22:18:10.367105 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.367079    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f89343/rate-test-model-f89343-kserve-699db75b4c-28t9g"]
Jul 13 22:18:10.406408 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.406367    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/873f2d3c-0e91-4087-94eb-eea64c9d354e-tls-certs\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:18:10.406408 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.406395    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-th2jr\" (UniqueName: \"kubernetes.io/projected/873f2d3c-0e91-4087-94eb-eea64c9d354e-kube-api-access-th2jr\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:18:10.406603 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.406417    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-home\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:18:10.406603 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.406435    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/873f2d3c-0e91-4087-94eb-eea64c9d354e-dshm\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:18:10.473503 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:10.473403    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="873f2d3c-0e91-4087-94eb-eea64c9d354e" path="/var/lib/kubelet/pods/873f2d3c-0e91-4087-94eb-eea64c9d354e/volumes"
Jul 13 22:18:11.485367 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.485340    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:11.504585 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.504553    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7605-139927600976928_05c61939-b5d0-494c-835e-dbf4e9ce0199/test-curl-7605-139927600976928/0.log"
Jul 13 22:18:11.526684 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.526638    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7605-139927600976928"]
Jul 13 22:18:11.529876 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.529849    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7605-139927600976928"]
Jul 13 22:18:11.624621 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.624515    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fms2\" (UniqueName: \"kubernetes.io/projected/05c61939-b5d0-494c-835e-dbf4e9ce0199-kube-api-access-8fms2\") pod \"05c61939-b5d0-494c-835e-dbf4e9ce0199\" (UID: \"05c61939-b5d0-494c-835e-dbf4e9ce0199\") "
Jul 13 22:18:11.626750 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.626722    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05c61939-b5d0-494c-835e-dbf4e9ce0199-kube-api-access-8fms2" (OuterVolumeSpecName: "kube-api-access-8fms2") pod "05c61939-b5d0-494c-835e-dbf4e9ce0199" (UID: "05c61939-b5d0-494c-835e-dbf4e9ce0199"). InnerVolumeSpecName "kube-api-access-8fms2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:18:11.657169 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.657131    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7605-139927600976928"]
Jul 13 22:18:11.657572 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.657555    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="873f2d3c-0e91-4087-94eb-eea64c9d354e" containerName="main"
Jul 13 22:18:11.657676 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.657574    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="873f2d3c-0e91-4087-94eb-eea64c9d354e" containerName="main"
Jul 13 22:18:11.657676 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.657619    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05c61939-b5d0-494c-835e-dbf4e9ce0199" containerName="test-curl-7605-139927600976928"
Jul 13 22:18:11.657676 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.657629    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="05c61939-b5d0-494c-835e-dbf4e9ce0199" containerName="test-curl-7605-139927600976928"
Jul 13 22:18:11.657830 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.657751    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="05c61939-b5d0-494c-835e-dbf4e9ce0199" containerName="test-curl-7605-139927600976928"
Jul 13 22:18:11.657830 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.657768    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="873f2d3c-0e91-4087-94eb-eea64c9d354e" containerName="main"
Jul 13 22:18:11.660892 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.660869    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:11.667339 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.667315    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-139927600976928"]
Jul 13 22:18:11.725525 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.725488    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8xpb\" (UniqueName: \"kubernetes.io/projected/7413b860-051c-4e1a-98f2-ba88c1202464-kube-api-access-k8xpb\") pod \"test-curl-7605-139927600976928\" (UID: \"7413b860-051c-4e1a-98f2-ba88c1202464\") " pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:11.725699 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.725552    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fms2\" (UniqueName: \"kubernetes.io/projected/05c61939-b5d0-494c-835e-dbf4e9ce0199-kube-api-access-8fms2\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:18:11.826569 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.826525    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8xpb\" (UniqueName: \"kubernetes.io/projected/7413b860-051c-4e1a-98f2-ba88c1202464-kube-api-access-k8xpb\") pod \"test-curl-7605-139927600976928\" (UID: \"7413b860-051c-4e1a-98f2-ba88c1202464\") " pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:11.834787 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.834758    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8xpb\" (UniqueName: \"kubernetes.io/projected/7413b860-051c-4e1a-98f2-ba88c1202464-kube-api-access-k8xpb\") pod \"test-curl-7605-139927600976928\" (UID: \"7413b860-051c-4e1a-98f2-ba88c1202464\") " pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:11.971774 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:11.971686    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:12.108949 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:12.108882    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-139927600976928"]
Jul 13 22:18:12.111285 ip-10-0-140-98 kubenswrapper[2564]: W0713 22:18:12.111256    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7413b860_051c_4e1a_98f2_ba88c1202464.slice/crio-832a305fe326d7dadd849a3e995f78dbe4d930b1eab42b5fb22bbec4a9d1577f WatchSource:0}: Error finding container 832a305fe326d7dadd849a3e995f78dbe4d930b1eab42b5fb22bbec4a9d1577f: Status 404 returned error can't find the container with id 832a305fe326d7dadd849a3e995f78dbe4d930b1eab42b5fb22bbec4a9d1577f
Jul 13 22:18:12.352223 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:12.352197    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:12.352411 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:12.352245    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e1fd5d3baa21ab9e63c468a78330bfacde5ada5fdc1fb1186aae68ecf13b20d"
Jul 13 22:18:12.353355 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:12.353332    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-139927600976928" event={"ID":"7413b860-051c-4e1a-98f2-ba88c1202464","Type":"ContainerStarted","Data":"832a305fe326d7dadd849a3e995f78dbe4d930b1eab42b5fb22bbec4a9d1577f"}
Jul 13 22:18:12.363642 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:12.363597    2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7605-139927600976928" oldPodUID="05c61939-b5d0-494c-835e-dbf4e9ce0199" podUID="7413b860-051c-4e1a-98f2-ba88c1202464"
Jul 13 22:18:12.475173 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:12.475142    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05c61939-b5d0-494c-835e-dbf4e9ce0199" path="/var/lib/kubelet/pods/05c61939-b5d0-494c-835e-dbf4e9ce0199/volumes"
Jul 13 22:18:13.360932 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:13.360897    2564 generic.go:356] "Generic (PLEG): container finished" podID="7413b860-051c-4e1a-98f2-ba88c1202464" containerID="90c3f1dfe800bd857a335da7c8e6ad88689d55e03ff10a479a432371b74e8dd9" exitCode=0
Jul 13 22:18:13.361350 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:13.360988    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-139927600976928" event={"ID":"7413b860-051c-4e1a-98f2-ba88c1202464","Type":"ContainerDied","Data":"90c3f1dfe800bd857a335da7c8e6ad88689d55e03ff10a479a432371b74e8dd9"}
Jul 13 22:18:14.501546 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:14.501521    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:14.521404 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:14.521368    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7605-139927600976928_7413b860-051c-4e1a-98f2-ba88c1202464/test-curl-7605-139927600976928/0.log"
Jul 13 22:18:14.547625 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:14.547594    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7605-139927600976928"]
Jul 13 22:18:14.549274 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:14.549247    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7605-139927600976928"]
Jul 13 22:18:14.549274 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:14.549265    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8xpb\" (UniqueName: \"kubernetes.io/projected/7413b860-051c-4e1a-98f2-ba88c1202464-kube-api-access-k8xpb\") pod \"7413b860-051c-4e1a-98f2-ba88c1202464\" (UID: \"7413b860-051c-4e1a-98f2-ba88c1202464\") "
Jul 13 22:18:14.551297 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:14.551275    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7413b860-051c-4e1a-98f2-ba88c1202464-kube-api-access-k8xpb" (OuterVolumeSpecName: "kube-api-access-k8xpb") pod "7413b860-051c-4e1a-98f2-ba88c1202464" (UID: "7413b860-051c-4e1a-98f2-ba88c1202464"). InnerVolumeSpecName "kube-api-access-k8xpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 22:18:14.650840 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:14.650759    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8xpb\" (UniqueName: \"kubernetes.io/projected/7413b860-051c-4e1a-98f2-ba88c1202464-kube-api-access-k8xpb\") on node \"ip-10-0-140-98.ec2.internal\" DevicePath \"\""
Jul 13 22:18:15.375934 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:15.375897    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:15.375934 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:15.375937    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="832a305fe326d7dadd849a3e995f78dbe4d930b1eab42b5fb22bbec4a9d1577f"
Jul 13 22:18:15.385801 ip-10-0-140-98 kubenswrapper[2564]: E0713 22:18:15.385771    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7605-139927600976928\" is forbidden: User \"system:node:ip-10-0-140-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-98.ec2.internal' and this object" podUID="7413b860-051c-4e1a-98f2-ba88c1202464" pod="opendatahub/test-curl-7605-139927600976928"
Jul 13 22:18:16.472724 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:18:16.472683    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7413b860-051c-4e1a-98f2-ba88c1202464" path="/var/lib/kubelet/pods/7413b860-051c-4e1a-98f2-ba88c1202464/volumes"
Jul 13 22:19:12.807511 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:12.807475    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5db8865497-6jvg6_b5916b8f-caaf-4c3e-91e2-4bfe6599a603/copy-manifests/0.log"
Jul 13 22:19:12.814598 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:12.814564    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5db8865497-6jvg6_b5916b8f-caaf-4c3e-91e2-4bfe6599a603/manager/0.log"
Jul 13 22:19:17.238984 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:17.238947    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jzdzz_8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8/manager/0.log"
Jul 13 22:19:18.062905 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:18.062871    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bff766844-g2vz4_a21ba74e-705e-47cf-a195-d9f3c75bd808/kube-auth-proxy/0.log"
Jul 13 22:19:18.426398 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:18.426303    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-9r9jt_6fefab67-ebed-402f-8f44-7553b119c7bb/payload-pre-processing/0.log"
Jul 13 22:19:18.550567 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:18.550531    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-9tdls_ca038f40-a926-4261-bdcf-6ef05e78ae26/payload-processing/0.log"
Jul 13 22:19:18.662545 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:18.662518    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 22:19:18.674523 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:18.674495    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xv5wb_15165918-15d0-4f74-b300-931b0c8d67f5/ovn-acl-logging/0.log"
Jul 13 22:19:19.257992 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:19.257966    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-5h6nl_53fc0589-601b-4aef-b34c-53c88378d2d2/main/0.log"
Jul 13 22:19:19.499663 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:19.499617    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-zc425_e7f3e02f-5cf2-4e0b-9213-cd2734c0f137/main/0.log"
Jul 13 22:19:19.620395 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:19.620315    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-w4gm7_256bfe03-ee61-4250-b399-bee682e8e28d/main/0.log"
Jul 13 22:19:26.438971 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:26.438940    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nkmbp_0cb23a38-df98-4abc-9ca3-43fe622a34c7/global-pull-secret-syncer/0.log"
Jul 13 22:19:26.560987 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:26.560956    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rrzxh_abaedb55-25f3-46db-a793-cc0a4f71ddd3/konnectivity-agent/0.log"
Jul 13 22:19:26.667036 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:26.666999    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-98.ec2.internal_09e44c48acdcebe81c75239f1dc1647b/haproxy/0.log"
Jul 13 22:19:31.689545 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:31.689512    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jzdzz_8c2a9b5e-e8c6-494c-a98d-bca8bd605ad8/manager/0.log"
Jul 13 22:19:33.239436 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.239402    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_adff7db5-7c76-44a4-b78a-14d7f4fbaa44/alertmanager/0.log"
Jul 13 22:19:33.263410 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.263379    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_adff7db5-7c76-44a4-b78a-14d7f4fbaa44/config-reloader/0.log"
Jul 13 22:19:33.288569 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.288541    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_adff7db5-7c76-44a4-b78a-14d7f4fbaa44/kube-rbac-proxy-web/0.log"
Jul 13 22:19:33.313781 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.313749    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_adff7db5-7c76-44a4-b78a-14d7f4fbaa44/kube-rbac-proxy/0.log"
Jul 13 22:19:33.336417 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.336390    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_adff7db5-7c76-44a4-b78a-14d7f4fbaa44/kube-rbac-proxy-metric/0.log"
Jul 13 22:19:33.373664 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.373632    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_adff7db5-7c76-44a4-b78a-14d7f4fbaa44/prom-label-proxy/0.log"
Jul 13 22:19:33.395370 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.395328    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_adff7db5-7c76-44a4-b78a-14d7f4fbaa44/init-config-reloader/0.log"
Jul 13 22:19:33.448683 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.448637    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-h685t_5c267f5a-72ce-4e51-89ad-676469abda4f/cluster-monitoring-operator/0.log"
Jul 13 22:19:33.489816 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.489729    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9rg85_ec8e284a-cc0b-4127-b7f0-b1475beae905/kube-state-metrics/0.log"
Jul 13 22:19:33.515735 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.515706    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9rg85_ec8e284a-cc0b-4127-b7f0-b1475beae905/kube-rbac-proxy-main/0.log"
Jul 13 22:19:33.555771 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.555733    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9rg85_ec8e284a-cc0b-4127-b7f0-b1475beae905/kube-rbac-proxy-self/0.log"
Jul 13 22:19:33.663384 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.663354    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qnb5j_23529af6-bf21-404d-bac0-4674d7f26383/node-exporter/0.log"
Jul 13 22:19:33.685105 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.685071    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qnb5j_23529af6-bf21-404d-bac0-4674d7f26383/kube-rbac-proxy/0.log"
Jul 13 22:19:33.707444 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:33.707418    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qnb5j_23529af6-bf21-404d-bac0-4674d7f26383/init-textfile/0.log"
Jul 13 22:19:34.154728 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:34.154695    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-th5wg_018b60a2-9e31-4968-bca8-e96049846740/prometheus-operator/0.log"
Jul 13 22:19:34.171325 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:34.171296    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-th5wg_018b60a2-9e31-4968-bca8-e96049846740/kube-rbac-proxy/0.log"
Jul 13 22:19:34.200082 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:34.200051    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-x98w5_827c7efa-4d29-4f71-a846-7dee96d13cb0/prometheus-operator-admission-webhook/0.log"
Jul 13 22:19:35.603921 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:35.603883    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-6dsxq_8cc99873-ff7d-4ebe-80d8-3147d30440ca/networking-console-plugin/0.log"
Jul 13 22:19:36.107860 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.107827    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"]
Jul 13 22:19:36.108201 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.108188    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7413b860-051c-4e1a-98f2-ba88c1202464" containerName="test-curl-7605-139927600976928"
Jul 13 22:19:36.108246 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.108203    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7413b860-051c-4e1a-98f2-ba88c1202464" containerName="test-curl-7605-139927600976928"
Jul 13 22:19:36.108292 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.108282    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7413b860-051c-4e1a-98f2-ba88c1202464" containerName="test-curl-7605-139927600976928"
Jul 13 22:19:36.111567 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.111546    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.113752 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.113733    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wjrg5\"/\"kube-root-ca.crt\""
Jul 13 22:19:36.113874 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.113765    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wjrg5\"/\"openshift-service-ca.crt\""
Jul 13 22:19:36.114480 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.114460    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wjrg5\"/\"default-dockercfg-wn26n\""
Jul 13 22:19:36.120967 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.120946    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"]
Jul 13 22:19:36.200225 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.200186    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6acbfb99-a165-439e-b6e2-f62911434cd1-proc\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.200431 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.200241    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6acbfb99-a165-439e-b6e2-f62911434cd1-lib-modules\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.200431 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.200296    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6acbfb99-a165-439e-b6e2-f62911434cd1-podres\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.200431 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.200343    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6acbfb99-a165-439e-b6e2-f62911434cd1-sys\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.200431 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.200375    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc55p\" (UniqueName: \"kubernetes.io/projected/6acbfb99-a165-439e-b6e2-f62911434cd1-kube-api-access-tc55p\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.301044 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.301005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6acbfb99-a165-439e-b6e2-f62911434cd1-lib-modules\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.301044 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.301047    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6acbfb99-a165-439e-b6e2-f62911434cd1-podres\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.301317 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.301088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6acbfb99-a165-439e-b6e2-f62911434cd1-sys\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.301317 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.301109    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc55p\" (UniqueName: \"kubernetes.io/projected/6acbfb99-a165-439e-b6e2-f62911434cd1-kube-api-access-tc55p\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.301317 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.301205    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6acbfb99-a165-439e-b6e2-f62911434cd1-lib-modules\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.301317 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.301219    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6acbfb99-a165-439e-b6e2-f62911434cd1-sys\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.301317 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.301203    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6acbfb99-a165-439e-b6e2-f62911434cd1-proc\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.301317 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.301227    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6acbfb99-a165-439e-b6e2-f62911434cd1-podres\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.301317 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.301253    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6acbfb99-a165-439e-b6e2-f62911434cd1-proc\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.308975 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.308954    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc55p\" (UniqueName: \"kubernetes.io/projected/6acbfb99-a165-439e-b6e2-f62911434cd1-kube-api-access-tc55p\") pod \"perf-node-gather-daemonset-dtk7l\" (UID: \"6acbfb99-a165-439e-b6e2-f62911434cd1\") " pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.422608 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.422504    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.567035 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.567001    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"]
Jul 13 22:19:36.570806 ip-10-0-140-98 kubenswrapper[2564]: W0713 22:19:36.570776    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6acbfb99_a165_439e_b6e2_f62911434cd1.slice/crio-324693027c44cd3707b4a80b6c11e917e8c7d68078986c639dd5908b8b86f514 WatchSource:0}: Error finding container 324693027c44cd3707b4a80b6c11e917e8c7d68078986c639dd5908b8b86f514: Status 404 returned error can't find the container with id 324693027c44cd3707b4a80b6c11e917e8c7d68078986c639dd5908b8b86f514
Jul 13 22:19:36.781817 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.781782    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6cf88bc554-vb7gt_00194f71-93a9-452e-8224-f1275f8e19a8/console/0.log"
Jul 13 22:19:36.792044 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.792014    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l" event={"ID":"6acbfb99-a165-439e-b6e2-f62911434cd1","Type":"ContainerStarted","Data":"5cc5634e5dd365dc3c80b93fee013b4e7c911ffda3e32d86a9f34a50871aa4f9"}
Jul 13 22:19:36.792220 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.792062    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l" event={"ID":"6acbfb99-a165-439e-b6e2-f62911434cd1","Type":"ContainerStarted","Data":"324693027c44cd3707b4a80b6c11e917e8c7d68078986c639dd5908b8b86f514"}
Jul 13 22:19:36.792220 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.792093    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:36.806785 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:36.806723    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l" podStartSLOduration=0.806705242 podStartE2EDuration="806.705242ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 22:19:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 22:19:36.805689444 +0000 UTC m=+3018.727738358" watchObservedRunningTime="2026-07-13 22:19:36.806705242 +0000 UTC m=+3018.728754159"
Jul 13 22:19:37.373121 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:37.373088    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-jcndp_7c0ac9d6-036a-40c0-b399-3270e38f959c/volume-data-source-validator/0.log"
Jul 13 22:19:38.423310 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:38.423276    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xcd5s_4418e5fc-2e14-430d-aee2-37c5eeae27bb/dns/0.log"
Jul 13 22:19:38.445558 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:38.445517    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xcd5s_4418e5fc-2e14-430d-aee2-37c5eeae27bb/kube-rbac-proxy/0.log"
Jul 13 22:19:38.470278 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:38.470252    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pt6c7_53433330-3378-4829-be11-5a219e621ef4/dns-node-resolver/0.log"
Jul 13 22:19:39.055826 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:39.055802    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4v4mk_c8726c19-73c9-4166-97a2-88f09f2bb28c/node-ca/0.log"
Jul 13 22:19:40.165224 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:40.165198    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bff766844-g2vz4_a21ba74e-705e-47cf-a195-d9f3c75bd808/kube-auth-proxy/0.log"
Jul 13 22:19:40.297579 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:40.297549    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-9r9jt_6fefab67-ebed-402f-8f44-7553b119c7bb/payload-pre-processing/0.log"
Jul 13 22:19:40.337502 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:40.337471    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-9tdls_ca038f40-a926-4261-bdcf-6ef05e78ae26/payload-processing/0.log"
Jul 13 22:19:41.095541 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:41.095511    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n2ltx_fc4ac07c-35b4-46be-8cda-9c35d1fc97bd/serve-healthcheck-canary/0.log"
Jul 13 22:19:41.620087 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:41.620059    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cnzqd_70c39abc-b85c-4453-95ad-d9ca15d1b706/kube-rbac-proxy/0.log"
Jul 13 22:19:41.641067 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:41.641040    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cnzqd_70c39abc-b85c-4453-95ad-d9ca15d1b706/exporter/0.log"
Jul 13 22:19:41.663413 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:41.663384    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cnzqd_70c39abc-b85c-4453-95ad-d9ca15d1b706/extractor/0.log"
Jul 13 22:19:42.806906 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:42.806879    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wjrg5/perf-node-gather-daemonset-dtk7l"
Jul 13 22:19:44.308112 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:44.308010    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5db8865497-6jvg6_b5916b8f-caaf-4c3e-91e2-4bfe6599a603/manager/0.log"
Jul 13 22:19:44.328613 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:44.328587    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5db8865497-6jvg6_b5916b8f-caaf-4c3e-91e2-4bfe6599a603/copy-manifests/0.log"
Jul 13 22:19:45.773516 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:45.773487    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-789595b4db-srkbn_f03eee92-cf1d-4b4e-9a89-60ce8ef9733f/manager/0.log"
Jul 13 22:19:46.436068 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:46.436040    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bff766844-g2vz4_a21ba74e-705e-47cf-a195-d9f3c75bd808/kube-auth-proxy/0.log"
Jul 13 22:19:46.549612 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:46.549532    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-9r9jt_6fefab67-ebed-402f-8f44-7553b119c7bb/payload-pre-processing/0.log"
Jul 13 22:19:46.578547 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:46.578520    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-9tdls_ca038f40-a926-4261-bdcf-6ef05e78ae26/payload-processing/0.log"
Jul 13 22:19:47.796740 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:47.796699    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bff766844-g2vz4_a21ba74e-705e-47cf-a195-d9f3c75bd808/kube-auth-proxy/0.log"
Jul 13 22:19:47.907510 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:47.907477    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-9r9jt_6fefab67-ebed-402f-8f44-7553b119c7bb/payload-pre-processing/0.log"
Jul 13 22:19:47.938513 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:47.938476    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-9tdls_ca038f40-a926-4261-bdcf-6ef05e78ae26/payload-processing/0.log"
Jul 13 22:19:50.699800 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:50.699770    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-twl8l_f4d535ca-9697-4610-9dba-cf0965688aaf/migrator/0.log"
Jul 13 22:19:50.719816 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:50.719784    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-twl8l_f4d535ca-9697-4610-9dba-cf0965688aaf/graceful-termination/0.log"
Jul 13 22:19:51.115931 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:51.115895    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-v9sjz_675ce41d-5f47-4029-aaf7-504ad5f2fc06/kube-storage-version-migrator-operator/1.log"
Jul 13 22:19:51.117435 ip-10-0-140-98 kubenswrapper[2564]: I0713 22:19:51.117415    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-v9sjz_675ce41d-5f47-4029-aaf7-504ad5f2fc06/kube-storage-version-migrator-operator/0.log"