Jul 15 19:13:27.005899 ip-10-0-143-114 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 15 19:13:27.005907 ip-10-0-143-114 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 15 19:13:27.005914 ip-10-0-143-114 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 15 19:13:27.006235 ip-10-0-143-114 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 15 19:13:36.528496 ip-10-0-143-114 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Jul 15 19:13:36.528511 ip-10-0-143-114 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c5d8ab95792643d980a547718219c786 --
Jul 15 19:15:27.813790 ip-10-0-143-114 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 19:15:28.315740 ip-10-0-143-114 systemd[1]: Started Kubernetes Kubelet.
Jul 15 19:15:28.323926 ip-10-0-143-114 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 19:15:28.323926 ip-10-0-143-114 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 19:15:28.323926 ip-10-0-143-114 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 19:15:28.323926 ip-10-0-143-114 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 19:15:28.323926 ip-10-0-143-114 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 19:15:28.323926 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.241357    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 19:15:28.323926 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246849    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:15:28.323926 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246861    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:15:28.323926 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246865    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:15:28.323926 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246868    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:15:28.323926 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246871    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:15:28.323926 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246874    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:15:28.323926 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246877    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246880    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246883    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246887    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246889    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246892    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246895    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246897    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246900    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246908    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246910    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246915    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246918    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246921    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246924    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246928    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246931    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246934    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246937    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246939    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:15:28.324749 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246941    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246944    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246947    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246949    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246952    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246955    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246957    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246960    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246962    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246965    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246968    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246970    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246973    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246975    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246978    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246980    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246982    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246985    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246987    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246990    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:15:28.325345 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246993    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246995    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.246998    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247001    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247003    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247006    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247008    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247010    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247013    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247016    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247018    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247021    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247023    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247026    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247028    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247030    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247033    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247036    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247039    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247041    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:15:28.325991 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247044    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247046    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247049    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247051    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247055    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247059    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247062    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247065    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247068    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247071    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247074    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247077    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247079    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247081    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247084    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247086    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247089    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247091    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247095    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:15:28.326617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247097    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247100    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247103    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247107    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247110    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247114    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247116    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247119    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247122    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247125    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247127    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247130    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247132    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247135    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247867    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247873    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247877    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247880    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247883    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247885    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:15:28.327228 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247888    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247890    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247894    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247898    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247900    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247903    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247905    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247908    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247911    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247913    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247916    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247919    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247923    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247925    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247928    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247930    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247933    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247936    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247938    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:15:28.328050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247941    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247943    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247948    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247952    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247955    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247957    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247961    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247964    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247966    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247969    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247972    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247976    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247979    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247981    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247984    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247987    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247990    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247993    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247996    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:15:28.328570 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.247998    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248000    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248003    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248006    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248008    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248011    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248014    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248016    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248019    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248022    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248024    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248027    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248030    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248033    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248036    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248038    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248041    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248044    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248046    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248049    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:15:28.329094 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248051    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248054    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248056    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248059    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248062    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248064    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248067    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248069    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248072    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248075    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248078    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248083    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248086    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248088    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248091    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248093    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248095    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248098    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248100    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248103    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:15:28.329704 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248105    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248107    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248110    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248113    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248116    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248119    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248122    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248124    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248127    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248133    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248136    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248139    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248143    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248146    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248149    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248530    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248535    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248538    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248541    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:15:28.330420 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248544    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248546    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248549    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248551    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248554    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248557    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248560    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248562    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248565    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248567    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248570    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248572    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248575    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248577    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248579    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248582    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248584    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248587    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248589    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248592    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:15:28.331032 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248595    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248597    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248601    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248604    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248606    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248610    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248613    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248617    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248621    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248623    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248627    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248630    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248632    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248635    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248638    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248640    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248642    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248646    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248648    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248651    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:15:28.331617 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248653    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248656    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248658    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248661    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248664    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248667    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248670    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248674    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248676    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248679    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248681    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248684    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248686    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248689    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248692    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248695    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248697    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248701    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248703    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248706    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:15:28.332162 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248709    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248711    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248713    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248716    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248718    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248720    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248723    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248726    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248728    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248731    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248734    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248736    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248739    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248742    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248744    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248747    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248749    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248753    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248755    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248758    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:15:28.332766 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248760    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248762    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248765    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248768    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248770    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248773    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248778    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248781    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248784    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248789    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248791    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248794    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248796    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248799    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.248802    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.248880    2560 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.248899    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.248911    2560 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.248924    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.248937    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.248946    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 19:15:28.333487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.248955    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.248962    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.248971    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.248983    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.248991    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.248998    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249003    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249009    2560 flags.go:64] FLAG: --cgroup-root=""
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249013    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249016    2560 flags.go:64] FLAG: --client-ca-file=""
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249019    2560 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249022    2560 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249026    2560 flags.go:64] FLAG: --cluster-domain=""
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249029    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249032    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249035    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249038    2560 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249043    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249046    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249049    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249054    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249057    2560 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249060    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249063    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 19:15:28.334309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249066    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249069    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249073    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249076    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249079    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249082    2560 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249085    2560 flags.go:64] FLAG: --enable-server="true"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249088    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249093    2560 flags.go:64] FLAG: --event-burst="100"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249096    2560 flags.go:64] FLAG: --event-qps="50"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249099    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249103    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249105    2560 flags.go:64] FLAG: --eviction-hard=""
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249109    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249112    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249115    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249118    2560 flags.go:64] FLAG: --eviction-soft=""
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249120    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249123    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249126    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249128    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249131    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249134    2560 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249137    2560 flags.go:64] FLAG: --feature-gates=""
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249141    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 19:15:28.335134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249145    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249148    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249151    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249158    2560 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249161    2560 flags.go:64] FLAG: --help="false"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249164    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249167    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249170    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249173    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249177    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249188    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249191    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249193    2560 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249196    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249199    2560 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249202    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249205    2560 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249208    2560 flags.go:64] FLAG: --kube-reserved=""
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249211    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249215    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249218    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249221    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249224    2560 flags.go:64] FLAG: --lock-file=""
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249226    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 19:15:28.335931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249229    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249232    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249237    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249240    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249242    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249245    2560 flags.go:64] FLAG: --logging-format="text"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249248    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249252    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249256    2560 flags.go:64] FLAG: --manifest-url=""
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249259    2560 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249263    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249268    2560 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249272    2560 flags.go:64] FLAG: --max-pods="110"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249275    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249278    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249281    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249283    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249287    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249289    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249292    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249300    2560 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249302    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249306    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 19:15:28.337284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249308    2560 flags.go:64] FLAG: --pod-cidr=""
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249311    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249317    2560 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249320    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249323    2560 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249326    2560 flags.go:64] FLAG: --port="10250"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249329    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249332    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0689d261e47f37468"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249335    2560 flags.go:64] FLAG: --qos-reserved=""
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249338    2560 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249341    2560 flags.go:64] FLAG: --register-node="true"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249344    2560 flags.go:64] FLAG: --register-with-taints=""
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249347    2560 flags.go:64] FLAG: --registry-burst="10"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249350    2560 flags.go:64] FLAG: --registry-qps="5"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249353    2560 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249355    2560 flags.go:64] FLAG: --reserved-memory=""
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249359    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249362    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249366    2560 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249369    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249372    2560 flags.go:64] FLAG: --runonce="false"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249375    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249378    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249381    2560 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249384    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 19:15:28.337915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249387    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249390    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249393    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249396    2560 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249399    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249401    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249404    2560 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249407    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249409    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249412    2560 flags.go:64] FLAG: --system-cgroups=""
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249415    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249420    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249423    2560 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249426    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249430    2560 flags.go:64] FLAG: --tls-min-version=""
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249433    2560 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249435    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249438    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249441    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249444    2560 flags.go:64] FLAG: --v="2"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249460    2560 flags.go:64] FLAG: --version="false"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249464    2560 flags.go:64] FLAG: --vmodule=""
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249467    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249470    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249583    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:15:28.616377 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249588    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249593    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249598    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249601    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249604    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249606    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249609    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249611    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249614    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249616    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249619    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249621    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249624    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249626    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249629    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249631    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249634    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249636    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249639    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:15:28.635476 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249643    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249646    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249650    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249664    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249670    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249675    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249680    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249682    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249685    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249688    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249691    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249694    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249696    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249700    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249702    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249706    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249709    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249711    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249714    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249716    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:15:28.636209 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249719    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249722    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249724    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249726    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249729    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249731    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249734    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249736    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249739    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249741    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249743    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249746    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249748    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249751    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249754    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249757    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249760    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249762    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249764    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:15:28.636950 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249767    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249769    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249772    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249775    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249777    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249779    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249782    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249785    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249789    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249792    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249794    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249797    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249799    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249802    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249805    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249807    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249809    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249812    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249814    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249818    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:15:28.637578 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249822    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249825    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249828    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249830    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249833    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249836    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249838    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249841    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249844    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249846    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249849    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249851    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249854    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249856    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249859    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249861    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249863    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249866    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249869    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:15:28.638255 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.249872    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.249879    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.257519    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.257536    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.258362    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.258376    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258470    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258476    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258481    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258485    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258489    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258494    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258498    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258502    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258507    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:15:28.638897 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258510    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258515    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258519    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258523    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258527    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258531    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258536    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258540    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258544    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258550    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258555    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258560    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258565    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258570    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258574    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258577    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258581    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258589    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258594    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:15:28.639429 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258599    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258603    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258606    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258610    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258614    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258618    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258623    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258627    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258630    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258634    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258638    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258642    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258645    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258649    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258653    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258657    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258661    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258665    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258669    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258673    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:15:28.640051 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258683    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258690    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258695    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258699    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258703    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258708    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258712    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258716    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258721    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258725    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258732    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258738    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258741    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258746    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258750    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258754    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258758    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258761    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258765    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:15:28.640746 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258769    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258773    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258776    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258782    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258786    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258791    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258794    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258798    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258802    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258806    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258810    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258814    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258817    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258821    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258825    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258830    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258834    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258838    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258842    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258845    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:15:28.641297 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258849    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258853    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258857    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258862    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258868    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258872    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258876    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258880    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258884    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258888    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258891    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.258895    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.258903    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259053    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259059    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 19:15:28.641922 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259064    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259068    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259073    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259077    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259082    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259086    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259091    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259095    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259099    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259103    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259108    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259112    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259116    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259121    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259126    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259130    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259133    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259138    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259141    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259145    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 19:15:28.642366 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259149    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259157    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259161    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259165    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259169    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259173    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259176    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259180    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259184    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259188    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259192    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259196    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259199    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259203    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259208    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259212    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259215    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259219    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259223    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259227    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 19:15:28.642987 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259231    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259236    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259241    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259246    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259250    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259255    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259259    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259264    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259269    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259273    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259276    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259280    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259284    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259291    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259295    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259299    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259303    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259307    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259311    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 19:15:28.643580 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259316    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259321    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259325    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259329    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259333    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259337    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259341    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259345    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259348    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259352    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259356    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259360    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259363    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259367    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259373    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259377    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259382    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259386    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259390    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 19:15:28.644169 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259394    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259399    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259403    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259408    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259412    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259415    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259419    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259426    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259430    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259433    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259437    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259441    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259445    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259463    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259467    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259471    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259475    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259479    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 19:15:28.644783 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:28.259483    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 19:15:28.645330 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.259490    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 19:15:28.645330 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.259615    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 19:15:28.645330 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.261763    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 19:15:28.645330 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.263116    2560 server.go:1019] "Starting client certificate rotation"
Jul 15 19:15:28.645330 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.263217    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 19:15:28.645330 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.263252    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 19:15:28.645330 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.265766    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 19:15:28.645330 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.275665    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 19:15:28.645330 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.282717    2560 log.go:25] "Validated CRI v1 runtime API"
Jul 15 19:15:28.645330 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.286865    2560 log.go:25] "Validated CRI v1 image API"
Jul 15 19:15:28.645330 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.289643    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 19:15:28.645330 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.292021    2560 fs.go:135] Filesystem UUIDs: map[0581e9f2-d3ad-4d7b-856e-cbdc2604d28d:/dev/nvme0n1p3 2aa6ca9e-0be8-407f-ad5b-97a919212464:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 15 19:15:28.645742 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.292035    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 19:15:28.645856 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.296089    2560 manager.go:216] Machine: {Timestamp:2026-07-15 19:15:28.295604134 +0000 UTC m=+0.365463522 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200149 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23f0e6312acde47e328739adc9c31e SystemUUID:ec23f0e6-312a-cde4-7e32-8739adc9c31e BootID:c5d8ab95-7926-43d9-80a5-47718219c786 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d0:4c:02:c6:53 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d0:4c:02:c6:53 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:d0:ac:94:f3:73 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 19:15:28.645856 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.296179    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 19:15:28.645856 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.296245    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 19:15:28.645856 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.299652    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 19:15:28.645856 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.299677    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-114.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 19:15:28.645856 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.300442    2560 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 19:15:28.645856 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.300466    2560 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 19:15:28.645856 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.300474    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.301096    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.301295    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.301904    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.305487    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.305630    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.305658    2560 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.305670    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.305681    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.305691    2560 kubelet.go:398] "Adding apiserver pod source"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.305699    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.306080    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.306096    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.308118    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.309626    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.309653    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.310914    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.310929    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.310935    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.310940    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.310949    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.310954    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.310963    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.310967    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.310979    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.310985    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.311000    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.311005    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.311728    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.311737    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.314822    2560 server.go:1268] "Started kubelet"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.314907    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.314905    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.314959    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.322094    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.327103    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n7cn7"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.327723    2560 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.327815    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.329874    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 19:15:28.875086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.329980    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n7cn7"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.330640    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.330711    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-114.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.329989    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-114.ec2.internal.18c28c8a5eba8710  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-114.ec2.internal,UID:ip-10-0-143-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-114.ec2.internal,},FirstTimestamp:2026-07-15 19:15:28.314795792 +0000 UTC m=+0.384655181,LastTimestamp:2026-07-15 19:15:28.314795792 +0000 UTC m=+0.384655181,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-114.ec2.internal,}"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.339617    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.339635    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.339734    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.340325    2560 factory.go:55] Registering systemd factory
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.340333    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.340347    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.340366    2560 factory.go:223] Registration of the systemd container factory successfully
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.340405    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.340438    2560 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.340444    2560 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.340650    2560 factory.go:153] Registering CRI-O factory
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.340664    2560 factory.go:223] Registration of the crio container factory successfully
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.340712    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.340735    2560 factory.go:103] Registering Raw factory
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.340749    2560 manager.go:1196] Started watching for new ooms in manager
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.341116    2560 manager.go:349] Starting recovery of all containers
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.341303    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.341434    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.351336    2560 manager.go:354] Recovery completed
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.356768    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.358540    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.358567    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.358576    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.359130    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.359144    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.359142    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:15:28.876361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.359159    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.359782    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.359807    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.359823    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.360353    2560 policy_none.go:49] "None policy: Start"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.360366    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.360375    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.361390    2560 policy_none.go:47] "Start"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.402975    2560 manager.go:337] "Starting Device Plugin manager"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.403097    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.403106    2560 server.go:87] "Starting device plugin registration server"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.403296    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.403307    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.403410    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.403499    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.403507    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.403894    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.403937    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.487626    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.488751    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.488784    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.488807    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.488842    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.491233    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.503773    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.504716    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.504739    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.504749    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.504769    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.510297    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.510315    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-114.ec2.internal\": node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.525165    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.590778    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-114.ec2.internal"]
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.590866    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.592429    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.592452    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.592483    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.593833    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:15:28.878081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.593940    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.593966    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.594569    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.594594    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.594605    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.594570    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.594663    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.594677    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.595928    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.595949    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.596678    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.596699    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.596717    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeHasSufficientPID"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.621069    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-114.ec2.internal\" not found" node="ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.624713    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-114.ec2.internal\" not found" node="ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.625643    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.726848    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.742324    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e766ce011d31786e76f8db29d81fbd7d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal\" (UID: \"e766ce011d31786e76f8db29d81fbd7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.742357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e766ce011d31786e76f8db29d81fbd7d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal\" (UID: \"e766ce011d31786e76f8db29d81fbd7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.742378    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef87f315cd7dcd1a6cbfa6284e94d05d-config\") pod \"kube-apiserver-proxy-ip-10-0-143-114.ec2.internal\" (UID: \"ef87f315cd7dcd1a6cbfa6284e94d05d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.827570    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.843036    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e766ce011d31786e76f8db29d81fbd7d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal\" (UID: \"e766ce011d31786e76f8db29d81fbd7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.879177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.843067    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e766ce011d31786e76f8db29d81fbd7d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal\" (UID: \"e766ce011d31786e76f8db29d81fbd7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.879837 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.843087    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef87f315cd7dcd1a6cbfa6284e94d05d-config\") pod \"kube-apiserver-proxy-ip-10-0-143-114.ec2.internal\" (UID: \"ef87f315cd7dcd1a6cbfa6284e94d05d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.879837 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.843141    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e766ce011d31786e76f8db29d81fbd7d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal\" (UID: \"e766ce011d31786e76f8db29d81fbd7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.879837 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.843151    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e766ce011d31786e76f8db29d81fbd7d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal\" (UID: \"e766ce011d31786e76f8db29d81fbd7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:28.879837 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.843208    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef87f315cd7dcd1a6cbfa6284e94d05d-config\") pod \"kube-apiserver-proxy-ip-10-0-143-114.ec2.internal\" (UID: \"ef87f315cd7dcd1a6cbfa6284e94d05d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:29.135385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.926079    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:29.135385 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:28.928372    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:29.135385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:28.928397    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:29.135385 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:29.031916    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:29.418346 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:29.138414    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:29.418346 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:29.244997    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:29.418346 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:29.262516    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 19:15:29.418346 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:29.262686    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 19:15:29.418346 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:29.336850    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 19:10:28 +0000 UTC" deadline="2028-04-20 11:58:56.314574192 +0000 UTC"
Jul 15 19:15:29.418346 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:29.336882    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15472h43m26.977695294s"
Jul 15 19:15:29.418346 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:29.345543    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:29.576488 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:29.445618    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:29.576488 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:29.546664    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-114.ec2.internal\" not found"
Jul 15 19:15:29.576488 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:29.555591    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:15:29.817165 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:29.622978    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef87f315cd7dcd1a6cbfa6284e94d05d.slice/crio-3fc287e3fd62cb7a7930912bb4a8a9869e2c624b4bc0532a296bfa80ac7f7586 WatchSource:0}: Error finding container 3fc287e3fd62cb7a7930912bb4a8a9869e2c624b4bc0532a296bfa80ac7f7586: Status 404 returned error can't find the container with id 3fc287e3fd62cb7a7930912bb4a8a9869e2c624b4bc0532a296bfa80ac7f7586
Jul 15 19:15:29.817165 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:29.629499    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:15:29.817165 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:29.630277    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 19:15:29.817165 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:29.640358    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:29.817165 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:29.648770    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:15:29.817165 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:29.651422    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 19:15:29.817165 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:29.652474    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-114.ec2.internal"
Jul 15 19:15:29.817165 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:29.660807    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.306770    2560 apiserver.go:52] "Watching apiserver"
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.318015    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.318389    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-114.ec2.internal","openshift-multus/multus-additional-cni-plugins-qm84p","openshift-multus/network-metrics-daemon-sk27q","openshift-network-diagnostics/network-check-target-7srhg","openshift-network-operator/iptables-alerter-vdwbm","openshift-ovn-kubernetes/ovnkube-node-z6vs2","kube-system/global-pull-secret-syncer-djwqm","kube-system/konnectivity-agent-l2wqc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q","openshift-cluster-node-tuning-operator/tuned-srs7q","openshift-dns/node-resolver-q2249","openshift-image-registry/node-ca-lgzhf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal","openshift-multus/multus-2dpn6"]
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.320167    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vdwbm"
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.321620    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.322782    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.322814    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h57sb\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.323046    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.323110    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q2249"
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.323174    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.323987    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.324371    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kj4zw\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.324743    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.325125    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.325397    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.325685    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.325692    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ptc52\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.325832    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.325856    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.325965    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.326030    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.326064    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.327439    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.328315    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.328749    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.328963    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 19:15:30.373826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.329172    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l2wqc"
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.329900    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j4zxk\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.330013    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.330131    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.330274    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.330514    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.330578    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.331214    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p5gk7\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.331240    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.331338    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.331459    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.331801    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.331814    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.331801    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wgck6\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.332280    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.333840    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.333906    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.334596    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jm2fs\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.334840    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.335010    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.336919    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lgzhf"
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.338492    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.339312    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.339428    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.339496    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.339623    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sb9tm\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.340194    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.340928    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.340978    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ncmhl\""
Jul 15 19:15:30.375168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.341836    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.350474    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361068    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/586d100d-7063-44bf-9004-0265d19b5bc5-cni-binary-copy\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361101    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/586d100d-7063-44bf-9004-0265d19b5bc5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361127    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6b44dee7-5ff5-4b39-b708-cc3e1c620758-ovn-node-metrics-cert\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361159    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvfdf\" (UniqueName: \"kubernetes.io/projected/cb2e4028-856d-4dba-bf82-1d056e6d2da2-kube-api-access-mvfdf\") pod \"network-metrics-daemon-sk27q\" (UID: \"cb2e4028-856d-4dba-bf82-1d056e6d2da2\") " pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361201    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-cni-bin\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361236    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-etc-kubernetes\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361253    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/31d06c6a-1375-4ecf-9428-4abd88ac7ab2-iptables-alerter-script\") pod \"iptables-alerter-vdwbm\" (UID: \"31d06c6a-1375-4ecf-9428-4abd88ac7ab2\") " pod="openshift-network-operator/iptables-alerter-vdwbm"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361282    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/586d100d-7063-44bf-9004-0265d19b5bc5-system-cni-dir\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361310    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-sys-fs\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361353    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-systemd-units\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361377    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-run-systemd\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361426    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-kubernetes\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.377056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361446    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-log-socket\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361495    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361547    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f63fb98-ad9f-4986-af0f-714ae207a152-host\") pod \"node-ca-lgzhf\" (UID: \"1f63fb98-ad9f-4986-af0f-714ae207a152\") " pod="openshift-image-registry/node-ca-lgzhf"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361584    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-run-netns\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361608    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-var-lib-kubelet\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361631    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-hostroot\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361650    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361689    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-run-k8s-cni-cncf-io\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361713    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-run-netns\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361731    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05fa4c6c-3a40-411f-8125-d086982a264b-agent-certs\") pod \"konnectivity-agent-l2wqc\" (UID: \"05fa4c6c-3a40-411f-8125-d086982a264b\") " pod="kube-system/konnectivity-agent-l2wqc"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361749    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-multus-cni-dir\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361769    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-var-lib-cni-bin\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361802    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19977ab6-3516-46be-931a-0dd0c38e808f-tmp-dir\") pod \"node-resolver-q2249\" (UID: \"19977ab6-3516-46be-931a-0dd0c38e808f\") " pod="openshift-dns/node-resolver-q2249"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361831    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-registration-dir\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.377989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361857    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-run-ovn-kubernetes\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361880    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6b44dee7-5ff5-4b39-b708-cc3e1c620758-env-overrides\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361900    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-tuned\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361914    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdrgn\" (UniqueName: \"kubernetes.io/projected/6b44dee7-5ff5-4b39-b708-cc3e1c620758-kube-api-access-zdrgn\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361937    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx25j\" (UniqueName: \"kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j\") pod \"network-check-target-7srhg\" (UID: \"955ee6c7-972b-4215-a5dc-ccdcb08f8da2\") " pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361953    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-sysconfig\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.361993    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c774fcf4-871c-46bf-8de9-fdb9faab88df-tmp\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362014    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f63fb98-ad9f-4986-af0f-714ae207a152-serviceca\") pod \"node-ca-lgzhf\" (UID: \"1f63fb98-ad9f-4986-af0f-714ae207a152\") " pod="openshift-image-registry/node-ca-lgzhf"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362037    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-os-release\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362062    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/209bcd45-469a-4a1d-9a46-3d512fcbd074-multus-daemon-config\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362085    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/586d100d-7063-44bf-9004-0265d19b5bc5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362099    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-run\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362113    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-lib-modules\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362133    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-var-lib-kubelet\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.378924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362154    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z7zq\" (UniqueName: \"kubernetes.io/projected/c774fcf4-871c-46bf-8de9-fdb9faab88df-kube-api-access-2z7zq\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.379828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362168    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-var-lib-cni-multus\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.379828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362181    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.379828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362195    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-socket-dir\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.379828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362212    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-device-dir\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.379828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362261    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-etc-openvswitch\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.379828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362294    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6b44dee7-5ff5-4b39-b708-cc3e1c620758-ovnkube-script-lib\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.379828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362331    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-run-ovn\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.379828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362368    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-node-log\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.379828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362396    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/586d100d-7063-44bf-9004-0265d19b5bc5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.379828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362428    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-host\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.379828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362454    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/209bcd45-469a-4a1d-9a46-3d512fcbd074-cni-binary-copy\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.379828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362501    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-multus-conf-dir\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.379828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362542    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/586d100d-7063-44bf-9004-0265d19b5bc5-os-release\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362586    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8tsh\" (UniqueName: \"kubernetes.io/projected/586d100d-7063-44bf-9004-0265d19b5bc5-kube-api-access-t8tsh\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362622    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txz5l\" (UniqueName: \"kubernetes.io/projected/19977ab6-3516-46be-931a-0dd0c38e808f-kube-api-access-txz5l\") pod \"node-resolver-q2249\" (UID: \"19977ab6-3516-46be-931a-0dd0c38e808f\") " pod="openshift-dns/node-resolver-q2249"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362650    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-etc-selinux\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362684    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-cni-netd\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362711    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/586d100d-7063-44bf-9004-0265d19b5bc5-cnibin\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362734    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-slash\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362758    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05fa4c6c-3a40-411f-8125-d086982a264b-konnectivity-ca\") pod \"konnectivity-agent-l2wqc\" (UID: \"05fa4c6c-3a40-411f-8125-d086982a264b\") " pod="kube-system/konnectivity-agent-l2wqc"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362782    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-systemd\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362807    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctrtm\" (UniqueName: \"kubernetes.io/projected/209bcd45-469a-4a1d-9a46-3d512fcbd074-kube-api-access-ctrtm\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362832    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/995ebd78-54f8-4755-8261-bdca88870130-dbus\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362856    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-sys\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362879    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzgc7\" (UniqueName: \"kubernetes.io/projected/1f63fb98-ad9f-4986-af0f-714ae207a152-kube-api-access-jzgc7\") pod \"node-ca-lgzhf\" (UID: \"1f63fb98-ad9f-4986-af0f-714ae207a152\") " pod="openshift-image-registry/node-ca-lgzhf"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.362987    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31d06c6a-1375-4ecf-9428-4abd88ac7ab2-host-slash\") pod \"iptables-alerter-vdwbm\" (UID: \"31d06c6a-1375-4ecf-9428-4abd88ac7ab2\") " pod="openshift-network-operator/iptables-alerter-vdwbm"
Jul 15 19:15:30.380518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363014    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19977ab6-3516-46be-931a-0dd0c38e808f-hosts-file\") pod \"node-resolver-q2249\" (UID: \"19977ab6-3516-46be-931a-0dd0c38e808f\") " pod="openshift-dns/node-resolver-q2249"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363039    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-kubelet\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363057    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-modprobe-d\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363082    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-system-cni-dir\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363107    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79qcz\" (UniqueName: \"kubernetes.io/projected/18c257f9-ec05-4f76-89db-949d5e22736e-kube-api-access-79qcz\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363133    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-run-openvswitch\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363158    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/995ebd78-54f8-4755-8261-bdca88870130-kubelet-config\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363185    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-sysctl-conf\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363230    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs\") pod \"network-metrics-daemon-sk27q\" (UID: \"cb2e4028-856d-4dba-bf82-1d056e6d2da2\") " pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363263    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-cnibin\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363289    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-run-multus-certs\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363315    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-var-lib-openvswitch\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363347    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6b44dee7-5ff5-4b39-b708-cc3e1c620758-ovnkube-config\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363371    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-sysctl-d\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.381487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363407    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-multus-socket-dir-parent\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.382315 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.363436    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttv7f\" (UniqueName: \"kubernetes.io/projected/31d06c6a-1375-4ecf-9428-4abd88ac7ab2-kube-api-access-ttv7f\") pod \"iptables-alerter-vdwbm\" (UID: \"31d06c6a-1375-4ecf-9428-4abd88ac7ab2\") " pod="openshift-network-operator/iptables-alerter-vdwbm"
Jul 15 19:15:30.382315 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.375111    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-b447h"
Jul 15 19:15:30.384019 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.383614    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-b447h"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.444400    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31d06c6a-1375-4ecf-9428-4abd88ac7ab2-host-slash\") pod \"iptables-alerter-vdwbm\" (UID: \"31d06c6a-1375-4ecf-9428-4abd88ac7ab2\") " pod="openshift-network-operator/iptables-alerter-vdwbm"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464216    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19977ab6-3516-46be-931a-0dd0c38e808f-hosts-file\") pod \"node-resolver-q2249\" (UID: \"19977ab6-3516-46be-931a-0dd0c38e808f\") " pod="openshift-dns/node-resolver-q2249"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464245    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-kubelet\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464269    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-modprobe-d\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464277    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31d06c6a-1375-4ecf-9428-4abd88ac7ab2-host-slash\") pod \"iptables-alerter-vdwbm\" (UID: \"31d06c6a-1375-4ecf-9428-4abd88ac7ab2\") " pod="openshift-network-operator/iptables-alerter-vdwbm"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464293    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-system-cni-dir\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464337    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79qcz\" (UniqueName: \"kubernetes.io/projected/18c257f9-ec05-4f76-89db-949d5e22736e-kube-api-access-79qcz\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464364    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-run-openvswitch\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464366    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19977ab6-3516-46be-931a-0dd0c38e808f-hosts-file\") pod \"node-resolver-q2249\" (UID: \"19977ab6-3516-46be-931a-0dd0c38e808f\") " pod="openshift-dns/node-resolver-q2249"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464371    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-system-cni-dir\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464410    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-run-openvswitch\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464410    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/995ebd78-54f8-4755-8261-bdca88870130-kubelet-config\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464428    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-kubelet\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.561698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464423    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-modprobe-d\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464446    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-sysctl-conf\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464452    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/995ebd78-54f8-4755-8261-bdca88870130-kubelet-config\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464485    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs\") pod \"network-metrics-daemon-sk27q\" (UID: \"cb2e4028-856d-4dba-bf82-1d056e6d2da2\") " pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464511    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-cnibin\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464531    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-run-multus-certs\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464555    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-var-lib-openvswitch\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464576    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6b44dee7-5ff5-4b39-b708-cc3e1c620758-ovnkube-config\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464606    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-cnibin\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464601    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-sysctl-d\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464626    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-sysctl-conf\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.464659    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464664    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-var-lib-openvswitch\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464666    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-run-multus-certs\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.563343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464664    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-multus-socket-dir-parent\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.564456 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.464727    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs podName:cb2e4028-856d-4dba-bf82-1d056e6d2da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:30.96470447 +0000 UTC m=+3.034563851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs") pod "network-metrics-daemon-sk27q" (UID: "cb2e4028-856d-4dba-bf82-1d056e6d2da2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:15:30.564456 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464783    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-sysctl-d\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.564456 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464784    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-multus-socket-dir-parent\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.564456 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464815    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttv7f\" (UniqueName: \"kubernetes.io/projected/31d06c6a-1375-4ecf-9428-4abd88ac7ab2-kube-api-access-ttv7f\") pod \"iptables-alerter-vdwbm\" (UID: \"31d06c6a-1375-4ecf-9428-4abd88ac7ab2\") " pod="openshift-network-operator/iptables-alerter-vdwbm"
Jul 15 19:15:30.564456 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464841    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/586d100d-7063-44bf-9004-0265d19b5bc5-cni-binary-copy\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.564456 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464867    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/586d100d-7063-44bf-9004-0265d19b5bc5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.564456 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.464894    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6b44dee7-5ff5-4b39-b708-cc3e1c620758-ovn-node-metrics-cert\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.564456 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465050    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/586d100d-7063-44bf-9004-0265d19b5bc5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.564456 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465086    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvfdf\" (UniqueName: \"kubernetes.io/projected/cb2e4028-856d-4dba-bf82-1d056e6d2da2-kube-api-access-mvfdf\") pod \"network-metrics-daemon-sk27q\" (UID: \"cb2e4028-856d-4dba-bf82-1d056e6d2da2\") " pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:30.564456 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465113    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-cni-bin\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.564456 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465138    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-etc-kubernetes\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.564456 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465167    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/31d06c6a-1375-4ecf-9428-4abd88ac7ab2-iptables-alerter-script\") pod \"iptables-alerter-vdwbm\" (UID: \"31d06c6a-1375-4ecf-9428-4abd88ac7ab2\") " pod="openshift-network-operator/iptables-alerter-vdwbm"
Jul 15 19:15:30.564456 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465194    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/586d100d-7063-44bf-9004-0265d19b5bc5-system-cni-dir\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465221    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-sys-fs\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465229    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465240    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6b44dee7-5ff5-4b39-b708-cc3e1c620758-ovnkube-config\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465246    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-systemd-units\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465250    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-etc-kubernetes\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465297    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-run-systemd\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465313    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/586d100d-7063-44bf-9004-0265d19b5bc5-system-cni-dir\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465321    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-kubernetes\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465318    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-cni-bin\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465386    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-systemd-units\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465393    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-kubernetes\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465402    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-run-systemd\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465421    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-log-socket\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465432    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-sys-fs\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.565269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465439    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/586d100d-7063-44bf-9004-0265d19b5bc5-cni-binary-copy\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465450    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465478    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f63fb98-ad9f-4986-af0f-714ae207a152-host\") pod \"node-ca-lgzhf\" (UID: \"1f63fb98-ad9f-4986-af0f-714ae207a152\") " pod="openshift-image-registry/node-ca-lgzhf"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465474    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-log-socket\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465527    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-run-netns\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465555    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-var-lib-kubelet\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465582    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-hostroot\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465603    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-run-netns\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465608    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465607    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f63fb98-ad9f-4986-af0f-714ae207a152-host\") pod \"node-ca-lgzhf\" (UID: \"1f63fb98-ad9f-4986-af0f-714ae207a152\") " pod="openshift-image-registry/node-ca-lgzhf"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465618    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-var-lib-kubelet\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465540    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465645    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-run-k8s-cni-cncf-io\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465645    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-hostroot\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465686    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-run-netns\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.566343 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.465697    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465709    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05fa4c6c-3a40-411f-8125-d086982a264b-agent-certs\") pod \"konnectivity-agent-l2wqc\" (UID: \"05fa4c6c-3a40-411f-8125-d086982a264b\") " pod="kube-system/konnectivity-agent-l2wqc"
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465684    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-run-k8s-cni-cncf-io\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465720    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-run-netns\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465729    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-multus-cni-dir\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.465749    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret podName:995ebd78-54f8-4755-8261-bdca88870130 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:30.965731102 +0000 UTC m=+3.035590485 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret") pod "global-pull-secret-syncer-djwqm" (UID: "995ebd78-54f8-4755-8261-bdca88870130") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465783    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-var-lib-cni-bin\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465830    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19977ab6-3516-46be-931a-0dd0c38e808f-tmp-dir\") pod \"node-resolver-q2249\" (UID: \"19977ab6-3516-46be-931a-0dd0c38e808f\") " pod="openshift-dns/node-resolver-q2249"
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465835    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/31d06c6a-1375-4ecf-9428-4abd88ac7ab2-iptables-alerter-script\") pod \"iptables-alerter-vdwbm\" (UID: \"31d06c6a-1375-4ecf-9428-4abd88ac7ab2\") " pod="openshift-network-operator/iptables-alerter-vdwbm"
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465840    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-multus-cni-dir\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465855    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-registration-dir\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465882    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-run-ovn-kubernetes\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465896    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-var-lib-cni-bin\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465907    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6b44dee7-5ff5-4b39-b708-cc3e1c620758-env-overrides\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.567276 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465929    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-registration-dir\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465944    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-run-ovn-kubernetes\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465932    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-tuned\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.465985    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdrgn\" (UniqueName: \"kubernetes.io/projected/6b44dee7-5ff5-4b39-b708-cc3e1c620758-kube-api-access-zdrgn\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.466013    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx25j\" (UniqueName: \"kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j\") pod \"network-check-target-7srhg\" (UID: \"955ee6c7-972b-4215-a5dc-ccdcb08f8da2\") " pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.466036    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-sysconfig\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.466059    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c774fcf4-871c-46bf-8de9-fdb9faab88df-tmp\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.466084    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f63fb98-ad9f-4986-af0f-714ae207a152-serviceca\") pod \"node-ca-lgzhf\" (UID: \"1f63fb98-ad9f-4986-af0f-714ae207a152\") " pod="openshift-image-registry/node-ca-lgzhf"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.466108    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-os-release\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.466110    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19977ab6-3516-46be-931a-0dd0c38e808f-tmp-dir\") pod \"node-resolver-q2249\" (UID: \"19977ab6-3516-46be-931a-0dd0c38e808f\") " pod="openshift-dns/node-resolver-q2249"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.466158    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/209bcd45-469a-4a1d-9a46-3d512fcbd074-multus-daemon-config\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.466159    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-sysconfig\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.466172    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-os-release\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.466929    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6b44dee7-5ff5-4b39-b708-cc3e1c620758-env-overrides\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.466977    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/586d100d-7063-44bf-9004-0265d19b5bc5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.568133 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467005    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-run\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467059    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-lib-modules\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467085    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-var-lib-kubelet\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467112    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z7zq\" (UniqueName: \"kubernetes.io/projected/c774fcf4-871c-46bf-8de9-fdb9faab88df-kube-api-access-2z7zq\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467138    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-var-lib-cni-multus\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467164    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-run\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467176    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1f63fb98-ad9f-4986-af0f-714ae207a152-serviceca\") pod \"node-ca-lgzhf\" (UID: \"1f63fb98-ad9f-4986-af0f-714ae207a152\") " pod="openshift-image-registry/node-ca-lgzhf"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467231    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467166    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467272    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-lib-modules\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467280    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/209bcd45-469a-4a1d-9a46-3d512fcbd074-multus-daemon-config\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467273    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-socket-dir\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467328    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-host-var-lib-cni-multus\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467333    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-device-dir\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467362    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-etc-openvswitch\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.569053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467388    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-var-lib-kubelet\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467359    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-socket-dir\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467390    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6b44dee7-5ff5-4b39-b708-cc3e1c620758-ovnkube-script-lib\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467439    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-run-ovn\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467445    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-etc-openvswitch\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467462    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-node-log\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467489    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/586d100d-7063-44bf-9004-0265d19b5bc5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467515    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-host\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467558    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/209bcd45-469a-4a1d-9a46-3d512fcbd074-cni-binary-copy\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467582    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-multus-conf-dir\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467582    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-run-ovn\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467405    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-device-dir\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467608    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/586d100d-7063-44bf-9004-0265d19b5bc5-os-release\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467633    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8tsh\" (UniqueName: \"kubernetes.io/projected/586d100d-7063-44bf-9004-0265d19b5bc5-kube-api-access-t8tsh\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.571195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467649    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-host\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467659    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txz5l\" (UniqueName: \"kubernetes.io/projected/19977ab6-3516-46be-931a-0dd0c38e808f-kube-api-access-txz5l\") pod \"node-resolver-q2249\" (UID: \"19977ab6-3516-46be-931a-0dd0c38e808f\") " pod="openshift-dns/node-resolver-q2249"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467685    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-etc-selinux\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467690    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-node-log\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467711    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-cni-netd\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467745    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/586d100d-7063-44bf-9004-0265d19b5bc5-cnibin\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467770    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-slash\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467796    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05fa4c6c-3a40-411f-8125-d086982a264b-konnectivity-ca\") pod \"konnectivity-agent-l2wqc\" (UID: \"05fa4c6c-3a40-411f-8125-d086982a264b\") " pod="kube-system/konnectivity-agent-l2wqc"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467821    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-systemd\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467844    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctrtm\" (UniqueName: \"kubernetes.io/projected/209bcd45-469a-4a1d-9a46-3d512fcbd074-kube-api-access-ctrtm\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467870    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/995ebd78-54f8-4755-8261-bdca88870130-dbus\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467894    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-sys\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467919    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzgc7\" (UniqueName: \"kubernetes.io/projected/1f63fb98-ad9f-4986-af0f-714ae207a152-kube-api-access-jzgc7\") pod \"node-ca-lgzhf\" (UID: \"1f63fb98-ad9f-4986-af0f-714ae207a152\") " pod="openshift-image-registry/node-ca-lgzhf"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.467975    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/18c257f9-ec05-4f76-89db-949d5e22736e-etc-selinux\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.572018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.468037    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6b44dee7-5ff5-4b39-b708-cc3e1c620758-ovnkube-script-lib\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.468135    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/586d100d-7063-44bf-9004-0265d19b5bc5-os-release\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.468154    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/586d100d-7063-44bf-9004-0265d19b5bc5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.468156    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/586d100d-7063-44bf-9004-0265d19b5bc5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.468192    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/209bcd45-469a-4a1d-9a46-3d512fcbd074-multus-conf-dir\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.468216    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/586d100d-7063-44bf-9004-0265d19b5bc5-cnibin\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.468252    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-systemd\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.468265    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-cni-netd\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.468307    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b44dee7-5ff5-4b39-b708-cc3e1c620758-host-slash\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.468360    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c774fcf4-871c-46bf-8de9-fdb9faab88df-sys\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.468418    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/995ebd78-54f8-4755-8261-bdca88870130-dbus\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.468448    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/209bcd45-469a-4a1d-9a46-3d512fcbd074-cni-binary-copy\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.468812    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05fa4c6c-3a40-411f-8125-d086982a264b-konnectivity-ca\") pod \"konnectivity-agent-l2wqc\" (UID: \"05fa4c6c-3a40-411f-8125-d086982a264b\") " pod="kube-system/konnectivity-agent-l2wqc"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.469362    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c774fcf4-871c-46bf-8de9-fdb9faab88df-etc-tuned\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.469636    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6b44dee7-5ff5-4b39-b708-cc3e1c620758-ovn-node-metrics-cert\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.572915 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.469936    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05fa4c6c-3a40-411f-8125-d086982a264b-agent-certs\") pod \"konnectivity-agent-l2wqc\" (UID: \"05fa4c6c-3a40-411f-8125-d086982a264b\") " pod="kube-system/konnectivity-agent-l2wqc"
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.470059    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c774fcf4-871c-46bf-8de9-fdb9faab88df-tmp\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.487191    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttv7f\" (UniqueName: \"kubernetes.io/projected/31d06c6a-1375-4ecf-9428-4abd88ac7ab2-kube-api-access-ttv7f\") pod \"iptables-alerter-vdwbm\" (UID: \"31d06c6a-1375-4ecf-9428-4abd88ac7ab2\") " pod="openshift-network-operator/iptables-alerter-vdwbm"
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.487358    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvfdf\" (UniqueName: \"kubernetes.io/projected/cb2e4028-856d-4dba-bf82-1d056e6d2da2-kube-api-access-mvfdf\") pod \"network-metrics-daemon-sk27q\" (UID: \"cb2e4028-856d-4dba-bf82-1d056e6d2da2\") " pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.493065    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z7zq\" (UniqueName: \"kubernetes.io/projected/c774fcf4-871c-46bf-8de9-fdb9faab88df-kube-api-access-2z7zq\") pod \"tuned-srs7q\" (UID: \"c774fcf4-871c-46bf-8de9-fdb9faab88df\") " pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.493585    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txz5l\" (UniqueName: \"kubernetes.io/projected/19977ab6-3516-46be-931a-0dd0c38e808f-kube-api-access-txz5l\") pod \"node-resolver-q2249\" (UID: \"19977ab6-3516-46be-931a-0dd0c38e808f\") " pod="openshift-dns/node-resolver-q2249"
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.493598    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctrtm\" (UniqueName: \"kubernetes.io/projected/209bcd45-469a-4a1d-9a46-3d512fcbd074-kube-api-access-ctrtm\") pod \"multus-2dpn6\" (UID: \"209bcd45-469a-4a1d-9a46-3d512fcbd074\") " pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.494835    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79qcz\" (UniqueName: \"kubernetes.io/projected/18c257f9-ec05-4f76-89db-949d5e22736e-kube-api-access-79qcz\") pod \"aws-ebs-csi-driver-node-lml8q\" (UID: \"18c257f9-ec05-4f76-89db-949d5e22736e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.496258    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.496307    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.496331    2560 projected.go:196] Error preparing data for projected volume kube-api-access-qx25j for pod openshift-network-diagnostics/network-check-target-7srhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.496403    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j podName:955ee6c7-972b-4215-a5dc-ccdcb08f8da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:30.996384157 +0000 UTC m=+3.066243552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qx25j" (UniqueName: "kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j") pod "network-check-target-7srhg" (UID: "955ee6c7-972b-4215-a5dc-ccdcb08f8da2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.497001    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8tsh\" (UniqueName: \"kubernetes.io/projected/586d100d-7063-44bf-9004-0265d19b5bc5-kube-api-access-t8tsh\") pod \"multus-additional-cni-plugins-qm84p\" (UID: \"586d100d-7063-44bf-9004-0265d19b5bc5\") " pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.497825    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal" event={"ID":"e766ce011d31786e76f8db29d81fbd7d","Type":"ContainerStarted","Data":"427be4420e253bb37785b9b09313bd8b00b1fb325f9af5bc6d45a04ea8ffa88c"}
Jul 15 19:15:30.573768 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.498037    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzgc7\" (UniqueName: \"kubernetes.io/projected/1f63fb98-ad9f-4986-af0f-714ae207a152-kube-api-access-jzgc7\") pod \"node-ca-lgzhf\" (UID: \"1f63fb98-ad9f-4986-af0f-714ae207a152\") " pod="openshift-image-registry/node-ca-lgzhf"
Jul 15 19:15:30.574694 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.498339    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdrgn\" (UniqueName: \"kubernetes.io/projected/6b44dee7-5ff5-4b39-b708-cc3e1c620758-kube-api-access-zdrgn\") pod \"ovnkube-node-z6vs2\" (UID: \"6b44dee7-5ff5-4b39-b708-cc3e1c620758\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.574694 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.499527    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-114.ec2.internal" event={"ID":"ef87f315cd7dcd1a6cbfa6284e94d05d","Type":"ContainerStarted","Data":"3fc287e3fd62cb7a7930912bb4a8a9869e2c624b4bc0532a296bfa80ac7f7586"}
Jul 15 19:15:30.704758 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.642624    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vdwbm"
Jul 15 19:15:30.704758 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.650313    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qm84p"
Jul 15 19:15:30.704758 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.662131    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q2249"
Jul 15 19:15:30.704758 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.668292    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q"
Jul 15 19:15:30.704758 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.674441    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:30.704758 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.683357    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l2wqc"
Jul 15 19:15:30.704758 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.691375    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-srs7q"
Jul 15 19:15:30.704758 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.699518    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lgzhf"
Jul 15 19:15:30.707371 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.706636    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2dpn6"
Jul 15 19:15:31.002772 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.971766    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:31.002772 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:30.971878    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs\") pod \"network-metrics-daemon-sk27q\" (UID: \"cb2e4028-856d-4dba-bf82-1d056e6d2da2\") " pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:31.002772 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.971961    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:15:31.002772 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.971991    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:15:31.002772 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.972048    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs podName:cb2e4028-856d-4dba-bf82-1d056e6d2da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:31.972030967 +0000 UTC m=+4.041890344 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs") pod "network-metrics-daemon-sk27q" (UID: "cb2e4028-856d-4dba-bf82-1d056e6d2da2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:15:31.002772 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:30.972062    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret podName:995ebd78-54f8-4755-8261-bdca88870130 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:31.972056348 +0000 UTC m=+4.041915724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret") pod "global-pull-secret-syncer-djwqm" (UID: "995ebd78-54f8-4755-8261-bdca88870130") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:15:31.144901 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.072582    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx25j\" (UniqueName: \"kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j\") pod \"network-check-target-7srhg\" (UID: \"955ee6c7-972b-4215-a5dc-ccdcb08f8da2\") " pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:31.144901 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:31.072765    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:15:31.144901 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:31.072797    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:15:31.144901 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:31.072807    2560 projected.go:196] Error preparing data for projected volume kube-api-access-qx25j for pod openshift-network-diagnostics/network-check-target-7srhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:15:31.144901 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:31.072872    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j podName:955ee6c7-972b-4215-a5dc-ccdcb08f8da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:32.07285592 +0000 UTC m=+4.142715297 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qx25j" (UniqueName: "kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j") pod "network-check-target-7srhg" (UID: "955ee6c7-972b-4215-a5dc-ccdcb08f8da2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:15:31.354814 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:31.354776    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c257f9_ec05_4f76_89db_949d5e22736e.slice/crio-907c189e19b903a95dc113184680be0cb5d2d195393a330bed83ccc5395f10d4 WatchSource:0}: Error finding container 907c189e19b903a95dc113184680be0cb5d2d195393a330bed83ccc5395f10d4: Status 404 returned error can't find the container with id 907c189e19b903a95dc113184680be0cb5d2d195393a330bed83ccc5395f10d4
Jul 15 19:15:31.360012 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:31.359980    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc774fcf4_871c_46bf_8de9_fdb9faab88df.slice/crio-124df2545bf0461cce537fafb6ff009038e291860747a73832f4b965c9becfec WatchSource:0}: Error finding container 124df2545bf0461cce537fafb6ff009038e291860747a73832f4b965c9becfec: Status 404 returned error can't find the container with id 124df2545bf0461cce537fafb6ff009038e291860747a73832f4b965c9becfec
Jul 15 19:15:31.365236 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:31.364956    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f63fb98_ad9f_4986_af0f_714ae207a152.slice/crio-b937edd87175d0dcaf76acb837f37e8defbfb14cef989ed68bee54f1f7ab9ae7 WatchSource:0}: Error finding container b937edd87175d0dcaf76acb837f37e8defbfb14cef989ed68bee54f1f7ab9ae7: Status 404 returned error can't find the container with id b937edd87175d0dcaf76acb837f37e8defbfb14cef989ed68bee54f1f7ab9ae7
Jul 15 19:15:31.365236 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:31.365151    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209bcd45_469a_4a1d_9a46_3d512fcbd074.slice/crio-556be3cf166fa772e50e0e083e3a7207efd4e7a7265f83e8dc3cdf327786aaa8 WatchSource:0}: Error finding container 556be3cf166fa772e50e0e083e3a7207efd4e7a7265f83e8dc3cdf327786aaa8: Status 404 returned error can't find the container with id 556be3cf166fa772e50e0e083e3a7207efd4e7a7265f83e8dc3cdf327786aaa8
Jul 15 19:15:31.367185 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:31.366865    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b44dee7_5ff5_4b39_b708_cc3e1c620758.slice/crio-cfa3c6ff556f7b36da4e3c63d7b163eb84244f5aa7494703de3384b00ea789c6 WatchSource:0}: Error finding container cfa3c6ff556f7b36da4e3c63d7b163eb84244f5aa7494703de3384b00ea789c6: Status 404 returned error can't find the container with id cfa3c6ff556f7b36da4e3c63d7b163eb84244f5aa7494703de3384b00ea789c6
Jul 15 19:15:31.368307 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:31.368265    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod586d100d_7063_44bf_9004_0265d19b5bc5.slice/crio-6ccbaef979fb1ae032bfba339184c1ff903a203e57b4642afabc2772d79007a4 WatchSource:0}: Error finding container 6ccbaef979fb1ae032bfba339184c1ff903a203e57b4642afabc2772d79007a4: Status 404 returned error can't find the container with id 6ccbaef979fb1ae032bfba339184c1ff903a203e57b4642afabc2772d79007a4
Jul 15 19:15:31.369349 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:31.369326    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31d06c6a_1375_4ecf_9428_4abd88ac7ab2.slice/crio-1352ace73c6b7a53617bacaf32a7021b3a5b90b7db5ed4bb0ce9549a97b75715 WatchSource:0}: Error finding container 1352ace73c6b7a53617bacaf32a7021b3a5b90b7db5ed4bb0ce9549a97b75715: Status 404 returned error can't find the container with id 1352ace73c6b7a53617bacaf32a7021b3a5b90b7db5ed4bb0ce9549a97b75715
Jul 15 19:15:31.372852 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:31.372478    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05fa4c6c_3a40_411f_8125_d086982a264b.slice/crio-6eb030cdc2579738a59367ddcbace1a668aba933512ad031a38fda05396f0bdc WatchSource:0}: Error finding container 6eb030cdc2579738a59367ddcbace1a668aba933512ad031a38fda05396f0bdc: Status 404 returned error can't find the container with id 6eb030cdc2579738a59367ddcbace1a668aba933512ad031a38fda05396f0bdc
Jul 15 19:15:31.373554 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:15:31.373535    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19977ab6_3516_46be_931a_0dd0c38e808f.slice/crio-ca349c6a330faca07ede81b71afd1263f56568a48d3272aa17d30b75ab958fe2 WatchSource:0}: Error finding container ca349c6a330faca07ede81b71afd1263f56568a48d3272aa17d30b75ab958fe2: Status 404 returned error can't find the container with id ca349c6a330faca07ede81b71afd1263f56568a48d3272aa17d30b75ab958fe2
Jul 15 19:15:31.386071 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.386041    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:10:30 +0000 UTC" deadline="2028-04-02 06:28:42.060444244 +0000 UTC"
Jul 15 19:15:31.386071 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.386067    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15035h13m10.674381827s"
Jul 15 19:15:31.489070 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.489042    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:31.489194 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:31.489152    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:31.503001 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.502966    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q2249" event={"ID":"19977ab6-3516-46be-931a-0dd0c38e808f","Type":"ContainerStarted","Data":"ca349c6a330faca07ede81b71afd1263f56568a48d3272aa17d30b75ab958fe2"}
Jul 15 19:15:31.504315 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.504277    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qm84p" event={"ID":"586d100d-7063-44bf-9004-0265d19b5bc5","Type":"ContainerStarted","Data":"6ccbaef979fb1ae032bfba339184c1ff903a203e57b4642afabc2772d79007a4"}
Jul 15 19:15:31.505707 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.505642    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2" event={"ID":"6b44dee7-5ff5-4b39-b708-cc3e1c620758","Type":"ContainerStarted","Data":"cfa3c6ff556f7b36da4e3c63d7b163eb84244f5aa7494703de3384b00ea789c6"}
Jul 15 19:15:31.506907 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.506877    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lgzhf" event={"ID":"1f63fb98-ad9f-4986-af0f-714ae207a152","Type":"ContainerStarted","Data":"b937edd87175d0dcaf76acb837f37e8defbfb14cef989ed68bee54f1f7ab9ae7"}
Jul 15 19:15:31.508117 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.508085    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-srs7q" event={"ID":"c774fcf4-871c-46bf-8de9-fdb9faab88df","Type":"ContainerStarted","Data":"124df2545bf0461cce537fafb6ff009038e291860747a73832f4b965c9becfec"}
Jul 15 19:15:31.509412 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.509382    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q" event={"ID":"18c257f9-ec05-4f76-89db-949d5e22736e","Type":"ContainerStarted","Data":"907c189e19b903a95dc113184680be0cb5d2d195393a330bed83ccc5395f10d4"}
Jul 15 19:15:31.510505 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.510481    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l2wqc" event={"ID":"05fa4c6c-3a40-411f-8125-d086982a264b","Type":"ContainerStarted","Data":"6eb030cdc2579738a59367ddcbace1a668aba933512ad031a38fda05396f0bdc"}
Jul 15 19:15:31.511879 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.511854    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vdwbm" event={"ID":"31d06c6a-1375-4ecf-9428-4abd88ac7ab2","Type":"ContainerStarted","Data":"1352ace73c6b7a53617bacaf32a7021b3a5b90b7db5ed4bb0ce9549a97b75715"}
Jul 15 19:15:31.513165 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.513139    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2dpn6" event={"ID":"209bcd45-469a-4a1d-9a46-3d512fcbd074","Type":"ContainerStarted","Data":"556be3cf166fa772e50e0e083e3a7207efd4e7a7265f83e8dc3cdf327786aaa8"}
Jul 15 19:15:31.985417 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.985150    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs\") pod \"network-metrics-daemon-sk27q\" (UID: \"cb2e4028-856d-4dba-bf82-1d056e6d2da2\") " pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:31.986038 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:31.985475    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:31.986038 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:31.985485    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:15:31.986038 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:31.985568    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs podName:cb2e4028-856d-4dba-bf82-1d056e6d2da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:33.985545292 +0000 UTC m=+6.055404676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs") pod "network-metrics-daemon-sk27q" (UID: "cb2e4028-856d-4dba-bf82-1d056e6d2da2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:15:31.986038 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:31.985605    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:15:31.986038 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:31.985678    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret podName:995ebd78-54f8-4755-8261-bdca88870130 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:33.985661109 +0000 UTC m=+6.055520503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret") pod "global-pull-secret-syncer-djwqm" (UID: "995ebd78-54f8-4755-8261-bdca88870130") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:15:32.087470 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:32.087351    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx25j\" (UniqueName: \"kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j\") pod \"network-check-target-7srhg\" (UID: \"955ee6c7-972b-4215-a5dc-ccdcb08f8da2\") " pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:32.087653 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:32.087539    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:15:32.087653 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:32.087562    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:15:32.087653 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:32.087572    2560 projected.go:196] Error preparing data for projected volume kube-api-access-qx25j for pod openshift-network-diagnostics/network-check-target-7srhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:15:32.087653 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:32.087620    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j podName:955ee6c7-972b-4215-a5dc-ccdcb08f8da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:34.087606943 +0000 UTC m=+6.157466325 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qx25j" (UniqueName: "kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j") pod "network-check-target-7srhg" (UID: "955ee6c7-972b-4215-a5dc-ccdcb08f8da2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:15:32.386909 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:32.386863    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:10:30 +0000 UTC" deadline="2028-03-09 22:57:04.014939927 +0000 UTC"
Jul 15 19:15:32.386909 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:32.386907    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14475h41m31.628037242s"
Jul 15 19:15:32.492823 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:32.492795    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:32.492968 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:32.492942    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:32.493043 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:32.493030    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:32.493384 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:32.493356    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:32.541833 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:32.541792    2560 generic.go:356] "Generic (PLEG): container finished" podID="e766ce011d31786e76f8db29d81fbd7d" containerID="4e0c0809afe644f1d51a3cfea4482a5be26923e48e41c4d63e0f1dfe857a1ab3" exitCode=0
Jul 15 19:15:32.542009 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:32.541894    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal" event={"ID":"e766ce011d31786e76f8db29d81fbd7d","Type":"ContainerDied","Data":"4e0c0809afe644f1d51a3cfea4482a5be26923e48e41c4d63e0f1dfe857a1ab3"}
Jul 15 19:15:32.554340 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:32.554303    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-114.ec2.internal" event={"ID":"ef87f315cd7dcd1a6cbfa6284e94d05d","Type":"ContainerStarted","Data":"296e3edebf839b425bfc07225f970410b2e87e569ed75f19aa1ba276b6850e0b"}
Jul 15 19:15:33.490032 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:33.489496    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:33.490032 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:33.489642    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:33.578622 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:33.578586    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal" event={"ID":"e766ce011d31786e76f8db29d81fbd7d","Type":"ContainerStarted","Data":"34e39a2585e0a7e353b09f127d48415588a8ee87b7cb8c08f30f512b2fd130d2"}
Jul 15 19:15:33.596208 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:33.596148    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-114.ec2.internal" podStartSLOduration=3.596125893 podStartE2EDuration="3.596125893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:15:32.571289777 +0000 UTC m=+4.641149180" watchObservedRunningTime="2026-07-15 19:15:33.596125893 +0000 UTC m=+5.665985294"
Jul 15 19:15:34.007646 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:34.006832    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:34.007646 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:34.006927    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs\") pod \"network-metrics-daemon-sk27q\" (UID: \"cb2e4028-856d-4dba-bf82-1d056e6d2da2\") " pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:34.007646 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:34.007066    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:15:34.007646 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:34.007130    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs podName:cb2e4028-856d-4dba-bf82-1d056e6d2da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:38.007110663 +0000 UTC m=+10.076970043 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs") pod "network-metrics-daemon-sk27q" (UID: "cb2e4028-856d-4dba-bf82-1d056e6d2da2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:15:34.007646 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:34.007555    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:15:34.007646 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:34.007606    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret podName:995ebd78-54f8-4755-8261-bdca88870130 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:38.007590657 +0000 UTC m=+10.077450040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret") pod "global-pull-secret-syncer-djwqm" (UID: "995ebd78-54f8-4755-8261-bdca88870130") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:15:34.108109 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:34.107449    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx25j\" (UniqueName: \"kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j\") pod \"network-check-target-7srhg\" (UID: \"955ee6c7-972b-4215-a5dc-ccdcb08f8da2\") " pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:34.108109 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:34.107679    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:15:34.108109 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:34.107700    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:15:34.108109 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:34.107714    2560 projected.go:196] Error preparing data for projected volume kube-api-access-qx25j for pod openshift-network-diagnostics/network-check-target-7srhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:15:34.108109 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:34.107778    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j podName:955ee6c7-972b-4215-a5dc-ccdcb08f8da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:38.107759752 +0000 UTC m=+10.177619130 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qx25j" (UniqueName: "kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j") pod "network-check-target-7srhg" (UID: "955ee6c7-972b-4215-a5dc-ccdcb08f8da2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:15:34.492270 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:34.492172    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:34.492714 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:34.492313    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:34.492714 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:34.492659    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:34.492823 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:34.492782    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:35.489434 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:35.488970    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:35.489434 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:35.489099    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:36.489810 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:36.489705    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:36.489810 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:36.489728    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:36.490289 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:36.489867    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:36.490289 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:36.489991    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:37.489535 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:37.489496    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:37.489729 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:37.489636    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:38.043101 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:38.042496    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs\") pod \"network-metrics-daemon-sk27q\" (UID: \"cb2e4028-856d-4dba-bf82-1d056e6d2da2\") " pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:38.043101 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:38.042560    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:38.043101 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:38.042702    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:15:38.043101 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:38.042737    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:15:38.043101 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:38.042773    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret podName:995ebd78-54f8-4755-8261-bdca88870130 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:46.042753083 +0000 UTC m=+18.112612491 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret") pod "global-pull-secret-syncer-djwqm" (UID: "995ebd78-54f8-4755-8261-bdca88870130") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:15:38.043101 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:38.042792    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs podName:cb2e4028-856d-4dba-bf82-1d056e6d2da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:46.042778891 +0000 UTC m=+18.112638267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs") pod "network-metrics-daemon-sk27q" (UID: "cb2e4028-856d-4dba-bf82-1d056e6d2da2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:15:38.144508 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:38.143909    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx25j\" (UniqueName: \"kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j\") pod \"network-check-target-7srhg\" (UID: \"955ee6c7-972b-4215-a5dc-ccdcb08f8da2\") " pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:38.144508 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:38.144091    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:15:38.144508 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:38.144114    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:15:38.144508 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:38.144126    2560 projected.go:196] Error preparing data for projected volume kube-api-access-qx25j for pod openshift-network-diagnostics/network-check-target-7srhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:15:38.144508 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:38.144180    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j podName:955ee6c7-972b-4215-a5dc-ccdcb08f8da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:15:46.144163353 +0000 UTC m=+18.214022729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qx25j" (UniqueName: "kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j") pod "network-check-target-7srhg" (UID: "955ee6c7-972b-4215-a5dc-ccdcb08f8da2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:15:38.491163 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:38.490551    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:38.491163 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:38.490658    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:38.491163 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:38.491019    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:38.491163 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:38.491118    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:39.489479 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:39.489415    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:39.489895 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:39.489559    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:40.489850 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:40.489820    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:40.489850 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:40.489841    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:40.490365 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:40.489944    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:40.490365 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:40.490340    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:41.489685 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:41.489649    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:41.489877 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:41.489775    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:42.489079 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:42.489038    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:42.489257 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:42.489083    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:42.489257 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:42.489178    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:42.489368 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:42.489316    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:43.489933 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:43.489901    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:43.490318 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:43.490021    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:44.489776 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:44.489717    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:44.489943 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:44.489859    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:44.489943 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:44.489909    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:44.490364 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:44.490020    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:45.489645 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:45.489613    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:45.489860 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:45.489734    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:46.104267 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:46.104230    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs\") pod \"network-metrics-daemon-sk27q\" (UID: \"cb2e4028-856d-4dba-bf82-1d056e6d2da2\") " pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:46.104818 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:46.104295    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:46.104818 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:46.104417    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:15:46.104818 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:46.104439    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:15:46.104818 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:46.104516    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs podName:cb2e4028-856d-4dba-bf82-1d056e6d2da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:02.104494301 +0000 UTC m=+34.174353699 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs") pod "network-metrics-daemon-sk27q" (UID: "cb2e4028-856d-4dba-bf82-1d056e6d2da2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:15:46.104818 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:46.104537    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret podName:995ebd78-54f8-4755-8261-bdca88870130 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:02.104529327 +0000 UTC m=+34.174388702 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret") pod "global-pull-secret-syncer-djwqm" (UID: "995ebd78-54f8-4755-8261-bdca88870130") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:15:46.205650 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:46.205616    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx25j\" (UniqueName: \"kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j\") pod \"network-check-target-7srhg\" (UID: \"955ee6c7-972b-4215-a5dc-ccdcb08f8da2\") " pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:46.205827 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:46.205781    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:15:46.205827 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:46.205801    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:15:46.205827 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:46.205813    2560 projected.go:196] Error preparing data for projected volume kube-api-access-qx25j for pod openshift-network-diagnostics/network-check-target-7srhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:15:46.205936 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:46.205875    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j podName:955ee6c7-972b-4215-a5dc-ccdcb08f8da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:02.205857375 +0000 UTC m=+34.275716764 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qx25j" (UniqueName: "kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j") pod "network-check-target-7srhg" (UID: "955ee6c7-972b-4215-a5dc-ccdcb08f8da2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:15:46.489840 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:46.489806    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:46.490017 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:46.489806    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:46.490017 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:46.489946    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:46.490120 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:46.490025    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:47.489632 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:47.489596    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:47.490083 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:47.489726    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:48.489810 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:48.489778    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:48.490245 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:48.489872    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:48.490245 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:48.490092    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:48.490245 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:48.490207    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:49.489601 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:49.489570    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:49.489754 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:49.489669    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:50.489937 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.489644    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:50.490677 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.489699    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:50.490677 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:50.490033    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:50.490677 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:50.490103    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:50.613801 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.613707    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q" event={"ID":"18c257f9-ec05-4f76-89db-949d5e22736e","Type":"ContainerStarted","Data":"2f4510a89771bcd0d19a0a519b2c4369cc35074cef9f489cf4d79492a1517efe"}
Jul 15 19:15:50.615151 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.615124    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l2wqc" event={"ID":"05fa4c6c-3a40-411f-8125-d086982a264b","Type":"ContainerStarted","Data":"6106655fc1b80bc131befeb96e2957f9d939d4ab8627161103e1d8157720b0bf"}
Jul 15 19:15:50.616687 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.616660    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2dpn6" event={"ID":"209bcd45-469a-4a1d-9a46-3d512fcbd074","Type":"ContainerStarted","Data":"f7bd8a5fd9423d242b651e9cee56e6e6c99d5084d238d2765cf52bc572b52c7e"}
Jul 15 19:15:50.618146 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.618120    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q2249" event={"ID":"19977ab6-3516-46be-931a-0dd0c38e808f","Type":"ContainerStarted","Data":"2d7e97c034d1f671f76fc3254437c7ff403834c14f826a24f39cae39d136cea6"}
Jul 15 19:15:50.619601 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.619577    2560 generic.go:356] "Generic (PLEG): container finished" podID="586d100d-7063-44bf-9004-0265d19b5bc5" containerID="0176fc2899cdfe3804558176421af28e0bfcd7c357d9ddea19013d7c536ac3a8" exitCode=0
Jul 15 19:15:50.619706 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.619658    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qm84p" event={"ID":"586d100d-7063-44bf-9004-0265d19b5bc5","Type":"ContainerDied","Data":"0176fc2899cdfe3804558176421af28e0bfcd7c357d9ddea19013d7c536ac3a8"}
Jul 15 19:15:50.622689 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.622621    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:15:50.623016 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.622994    2560 generic.go:356] "Generic (PLEG): container finished" podID="6b44dee7-5ff5-4b39-b708-cc3e1c620758" containerID="c256a63a4e268721dad2f38b215e900fbed6c3c4eb619c79f595bb247a36dcdd" exitCode=1
Jul 15 19:15:50.623072 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.623058    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2" event={"ID":"6b44dee7-5ff5-4b39-b708-cc3e1c620758","Type":"ContainerStarted","Data":"6748a9c0161284e4bcc1e515f3f8757d341b83ea0e9b4f64e1f29285b2951fa3"}
Jul 15 19:15:50.623139 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.623127    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2" event={"ID":"6b44dee7-5ff5-4b39-b708-cc3e1c620758","Type":"ContainerStarted","Data":"3f3c02dd4696379d55d3b97ca27d059c3b516d71015e1611731c70396358bd01"}
Jul 15 19:15:50.623182 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.623174    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2" event={"ID":"6b44dee7-5ff5-4b39-b708-cc3e1c620758","Type":"ContainerStarted","Data":"743b4294589e9ee384083a60c3ee2e8ec9726819ac566c78399ff4ad5705ebdb"}
Jul 15 19:15:50.623215 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.623203    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2" event={"ID":"6b44dee7-5ff5-4b39-b708-cc3e1c620758","Type":"ContainerStarted","Data":"5714ba594e4bb1ca4f2c8d40be9eb0e51e9c89a37dfbe3b7d8ade50881322ee5"}
Jul 15 19:15:50.623256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.623230    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2" event={"ID":"6b44dee7-5ff5-4b39-b708-cc3e1c620758","Type":"ContainerDied","Data":"c256a63a4e268721dad2f38b215e900fbed6c3c4eb619c79f595bb247a36dcdd"}
Jul 15 19:15:50.623304 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.623263    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2" event={"ID":"6b44dee7-5ff5-4b39-b708-cc3e1c620758","Type":"ContainerStarted","Data":"48d9ddab20caad2a4e7c3a9efdf0ba6bdfd7f811f7e72339a879214665f3dd02"}
Jul 15 19:15:50.624495 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.624476    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lgzhf" event={"ID":"1f63fb98-ad9f-4986-af0f-714ae207a152","Type":"ContainerStarted","Data":"03e79537af6db30d9c68618af32ce1d190143bfd244931d56bb89c7067423fe5"}
Jul 15 19:15:50.625871 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.625854    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-srs7q" event={"ID":"c774fcf4-871c-46bf-8de9-fdb9faab88df","Type":"ContainerStarted","Data":"02cb9085a63a59bd31b15a026995c9877153d479f6c7c9db05de1e6444b848ef"}
Jul 15 19:15:50.630419 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.630382    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l2wqc" podStartSLOduration=3.351131734 podStartE2EDuration="21.630372366s" totalImagesPullingTime="18.279240632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:15:50.630233718 +0000 UTC m=+22.700093114" watchObservedRunningTime="2026-07-15 19:15:50.630372366 +0000 UTC m=+22.700231762"
Jul 15 19:15:50.630687 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.630662    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-114.ec2.internal" podStartSLOduration=21.630656964 podStartE2EDuration="21.630656964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:15:33.596987794 +0000 UTC m=+5.666847195" watchObservedRunningTime="2026-07-15 19:15:50.630656964 +0000 UTC m=+22.700516363"
Jul 15 19:15:50.647669 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.647626    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-srs7q" podStartSLOduration=3.356415373 podStartE2EDuration="21.64761696s" totalImagesPullingTime="18.291201587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:15:50.647614825 +0000 UTC m=+22.717474221" watchObservedRunningTime="2026-07-15 19:15:50.64761696 +0000 UTC m=+22.717476355"
Jul 15 19:15:50.663789 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.663748    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2dpn6" podStartSLOduration=3.2216500359999998 podStartE2EDuration="21.663736328s" totalImagesPullingTime="18.442086292s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:15:50.663504254 +0000 UTC m=+22.733363650" watchObservedRunningTime="2026-07-15 19:15:50.663736328 +0000 UTC m=+22.733595726"
Jul 15 19:15:50.697756 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.697717    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lgzhf" podStartSLOduration=3.409501905 podStartE2EDuration="21.697703969s" totalImagesPullingTime="18.288202064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:15:50.697547964 +0000 UTC m=+22.767407360" watchObservedRunningTime="2026-07-15 19:15:50.697703969 +0000 UTC m=+22.767563366"
Jul 15 19:15:50.713329 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.713291    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-q2249" podStartSLOduration=4.434177759 podStartE2EDuration="22.713279664s" totalImagesPullingTime="18.279101905s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:15:50.713225328 +0000 UTC m=+22.783084725" watchObservedRunningTime="2026-07-15 19:15:50.713279664 +0000 UTC m=+22.783139060"
Jul 15 19:15:50.806424 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.806397    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:50.872341 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:50.872280    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:51.009849 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:51.009802    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:51.110282 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:51.110251    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:51.489547 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:51.489520    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:51.489900 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:51.489624    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:51.620468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:51.620426    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:51.630782 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:51.630749    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vdwbm" event={"ID":"31d06c6a-1375-4ecf-9428-4abd88ac7ab2","Type":"ContainerStarted","Data":"6fcdfd3426d96f1c79475db4a6cd04049f980042bb5cb41fe70ad60758980b60"}
Jul 15 19:15:51.647011 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:51.646962    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vdwbm" podStartSLOduration=5.25331449 podStartE2EDuration="23.64694388s" totalImagesPullingTime="18.39362939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:15:51.646689498 +0000 UTC m=+23.716548895" watchObservedRunningTime="2026-07-15 19:15:51.64694388 +0000 UTC m=+23.716803279"
Jul 15 19:15:51.725730 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:51.725687    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:51.797126 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:51.797048    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:51.845379 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:51.845341    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:52.134891 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:52.134867    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:52.156197 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:52.156174    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 19:15:52.225183 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:52.225152    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:52.324140 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:52.324052    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:52.416715 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:52.416537    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T19:15:52.156192753Z","UUID":"76249858-6709-4f1e-b709-2ca9d69a32ca","Handler":null,"Name":"","Endpoint":""}
Jul 15 19:15:52.418324 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:52.418307    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 19:15:52.418437 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:52.418343    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 19:15:52.426359 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:52.426327    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:52.489279 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:52.489245    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:52.489434 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:52.489253    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:52.489434 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:52.489379    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:52.489543 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:52.489435    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:52.634364 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:52.634334    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q" event={"ID":"18c257f9-ec05-4f76-89db-949d5e22736e","Type":"ContainerStarted","Data":"e043892217d72873336fe3c5d8f54219c095f4dc0a74f9f60e62bb5425d4198a"}
Jul 15 19:15:53.489511 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:53.489322    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:53.489712 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:53.489587    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:53.638600 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:53.638506    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q" event={"ID":"18c257f9-ec05-4f76-89db-949d5e22736e","Type":"ContainerStarted","Data":"fedb5706aebd66ffe824c2b61e7ca9adb45764c367959a33f3b42a2bdf7a9b6e"}
Jul 15 19:15:53.641737 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:53.641718    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:15:53.642108 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:53.642086    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2" event={"ID":"6b44dee7-5ff5-4b39-b708-cc3e1c620758","Type":"ContainerStarted","Data":"a4aeb4785e261fc5518a1a67ba41d03200a1c5f3d5f0d8b5df38e5c13e21d15b"}
Jul 15 19:15:53.655173 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:53.655136    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lml8q" podStartSLOduration=3.981658984 podStartE2EDuration="25.655122369s" totalImagesPullingTime="21.673463385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:15:53.655028389 +0000 UTC m=+25.724887786" watchObservedRunningTime="2026-07-15 19:15:53.655122369 +0000 UTC m=+25.724981766"
Jul 15 19:15:53.702288 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:53.702255    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l2wqc"
Jul 15 19:15:53.702875 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:53.702858    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l2wqc"
Jul 15 19:15:53.956089 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:53.956018    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:54.012355 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:54.012328    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:54.490086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:54.490050    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:54.490254 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:54.490221    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:54.490306 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:54.490273    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:54.490426 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:54.490399    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:54.644807 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:54.644709    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l2wqc"
Jul 15 19:15:54.645328 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:54.645111    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l2wqc"
Jul 15 19:15:54.711810 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:54.711761    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:54.806310 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:54.806278    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:54.906991 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:54.906956    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:55.008683 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:55.008630    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:55.267438 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:55.267299    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:55.488961 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:55.488937    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:55.489095 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:55.489019    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:55.595147 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:55.595071    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:55.647867 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:55.647834    2560 generic.go:356] "Generic (PLEG): container finished" podID="586d100d-7063-44bf-9004-0265d19b5bc5" containerID="79d49579eba28c48864c7482ddfd6886f6c52a643d9844ed852eede36969d75c" exitCode=0
Jul 15 19:15:55.648699 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:55.647907    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qm84p" event={"ID":"586d100d-7063-44bf-9004-0265d19b5bc5","Type":"ContainerDied","Data":"79d49579eba28c48864c7482ddfd6886f6c52a643d9844ed852eede36969d75c"}
Jul 15 19:15:55.651320 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:55.651306    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:15:55.651724 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:55.651701    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2" event={"ID":"6b44dee7-5ff5-4b39-b708-cc3e1c620758","Type":"ContainerStarted","Data":"544149abd4ec9b3dab407ac3d95d7985d5c772860963b1232c5ccadc9affc42d"}
Jul 15 19:15:55.652075 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:55.652060    2560 scope.go:117] "RemoveContainer" containerID="c256a63a4e268721dad2f38b215e900fbed6c3c4eb619c79f595bb247a36dcdd"
Jul 15 19:15:55.819115 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:55.819081    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:55.920951 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:55.920889    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:56.023838 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:56.023801    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:56.091931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:56.091894    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:56.268237 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:56.268196    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:56.489729 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:56.489687    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:56.489965 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:56.489796    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:56.489965 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:56.489844    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:56.489965 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:56.489936    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:56.658334 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:56.658260    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:15:56.658784 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:56.658743    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2" event={"ID":"6b44dee7-5ff5-4b39-b708-cc3e1c620758","Type":"ContainerStarted","Data":"1e008c85b22b68682c8b6dd5fcb76ba4f840c673e4616885f5f1f78e63b47af7"}
Jul 15 19:15:56.659136 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:56.658977    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:56.659136 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:56.659058    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:56.659265 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:56.659247    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:56.674964 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:56.674943    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:56.675082 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:56.675032    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:15:56.685263 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:56.685229    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2" podStartSLOduration=9.257626897 podStartE2EDuration="27.685216574s" totalImagesPullingTime="18.427589677s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:15:56.684061171 +0000 UTC m=+28.753920567" watchObservedRunningTime="2026-07-15 19:15:56.685216574 +0000 UTC m=+28.755075965"
Jul 15 19:15:57.062465 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:57.062391    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:57.489344 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:57.489321    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:57.489515 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:57.489409    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:57.648372 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:57.648339    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:57.662587 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:57.662560    2560 generic.go:356] "Generic (PLEG): container finished" podID="586d100d-7063-44bf-9004-0265d19b5bc5" containerID="7d024907802b8aa432a4ec825cbf21307a0be118dbc33b1334f3aa7f76f40f26" exitCode=0
Jul 15 19:15:57.662989 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:57.662629    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qm84p" event={"ID":"586d100d-7063-44bf-9004-0265d19b5bc5","Type":"ContainerDied","Data":"7d024907802b8aa432a4ec825cbf21307a0be118dbc33b1334f3aa7f76f40f26"}
Jul 15 19:15:57.749048 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:57.748975    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:57.877072 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:57.877039    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:58.064511 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:58.064430    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:58.462047 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:58.461971    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:58.489917 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:58.489887    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:15:58.490058 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:58.489970    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:15:58.490105 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:58.490060    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:15:58.490173 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:58.490155    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:15:59.063259 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:59.063226    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:59.267220 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:59.267185    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:59.381811 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:59.381785    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:15:59.489290 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:59.489260    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:15:59.489476 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:15:59.489368    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:15:59.668862 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:59.668776    2560 generic.go:356] "Generic (PLEG): container finished" podID="586d100d-7063-44bf-9004-0265d19b5bc5" containerID="55241cb1013e5defb0a1208992e0b9406b6e3f9aa47446082f4ed0c1083640e6" exitCode=0
Jul 15 19:15:59.668862 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:15:59.668841    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qm84p" event={"ID":"586d100d-7063-44bf-9004-0265d19b5bc5","Type":"ContainerDied","Data":"55241cb1013e5defb0a1208992e0b9406b6e3f9aa47446082f4ed0c1083640e6"}
Jul 15 19:16:00.489896 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:00.489862    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:00.490335 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:00.489866    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:00.490335 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:00.490002    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:16:00.490335 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:00.490256    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:16:00.675387 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:00.675348    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:00.725060 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:00.725026    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:01.063839 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:01.063808    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:01.489126 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:01.489095    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:01.489287 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:01.489210    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:16:01.666035 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:01.666009    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:01.766469 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:01.766374    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:01.869696 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:01.869664    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:02.065478 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:02.065380    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:02.124352 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:02.124322    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:02.124529 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:02.124400    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs\") pod \"network-metrics-daemon-sk27q\" (UID: \"cb2e4028-856d-4dba-bf82-1d056e6d2da2\") " pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:02.124529 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:02.124485    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 19:16:02.124529 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:02.124500    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:16:02.124671 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:02.124549    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret podName:995ebd78-54f8-4755-8261-bdca88870130 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:34.124532973 +0000 UTC m=+66.194392361 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret") pod "global-pull-secret-syncer-djwqm" (UID: "995ebd78-54f8-4755-8261-bdca88870130") : object "kube-system"/"original-pull-secret" not registered
Jul 15 19:16:02.124671 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:02.124567    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs podName:cb2e4028-856d-4dba-bf82-1d056e6d2da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:34.124558612 +0000 UTC m=+66.194417990 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs") pod "network-metrics-daemon-sk27q" (UID: "cb2e4028-856d-4dba-bf82-1d056e6d2da2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 19:16:02.224944 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:02.224908    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx25j\" (UniqueName: \"kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j\") pod \"network-check-target-7srhg\" (UID: \"955ee6c7-972b-4215-a5dc-ccdcb08f8da2\") " pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:02.225117 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:02.225078    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 19:16:02.225117 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:02.225098    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 19:16:02.225117 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:02.225111    2560 projected.go:196] Error preparing data for projected volume kube-api-access-qx25j for pod openshift-network-diagnostics/network-check-target-7srhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:16:02.225262 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:02.225172    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j podName:955ee6c7-972b-4215-a5dc-ccdcb08f8da2 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:34.225153174 +0000 UTC m=+66.295012567 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qx25j" (UniqueName: "kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j") pod "network-check-target-7srhg" (UID: "955ee6c7-972b-4215-a5dc-ccdcb08f8da2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 19:16:02.464076 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:02.464001    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:02.493180 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:02.493154    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:02.493316 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:02.493155    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:02.493316 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:02.493281    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:16:02.493424 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:02.493342    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:16:02.894043 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:02.894006    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:03.267844 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:03.267807    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:03.465056 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:03.465027    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:03.489000 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:03.488970    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:03.489142 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:03.489085    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:16:03.681100 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:03.681062    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qm84p" event={"ID":"586d100d-7063-44bf-9004-0265d19b5bc5","Type":"ContainerStarted","Data":"0d9d754b45d608b2caed796abec5752e514d0b5dd6a60b004c36652fa94ebc79"}
Jul 15 19:16:03.860894 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:03.860821    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:04.062103 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:04.062076    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:04.271684 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:04.271658    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:04.489984 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:04.489948    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:04.490167 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:04.489954    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:04.490167 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:04.490081    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:16:04.490310 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:04.490185    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:16:04.685443 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:04.685362    2560 generic.go:356] "Generic (PLEG): container finished" podID="586d100d-7063-44bf-9004-0265d19b5bc5" containerID="0d9d754b45d608b2caed796abec5752e514d0b5dd6a60b004c36652fa94ebc79" exitCode=0
Jul 15 19:16:04.685443 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:04.685409    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qm84p" event={"ID":"586d100d-7063-44bf-9004-0265d19b5bc5","Type":"ContainerDied","Data":"0d9d754b45d608b2caed796abec5752e514d0b5dd6a60b004c36652fa94ebc79"}
Jul 15 19:16:05.263349 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:05.263152    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:05.461288 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:05.461264    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:05.489275 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:05.489257    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:05.489368 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:05.489351    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:16:05.659957 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:05.659877    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:05.690155 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:05.690128    2560 generic.go:356] "Generic (PLEG): container finished" podID="586d100d-7063-44bf-9004-0265d19b5bc5" containerID="1e7e62a1a7de229aa2397989c96264aca3e0ac58b70df7e2db2e5991560e47a6" exitCode=0
Jul 15 19:16:05.690280 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:05.690177    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qm84p" event={"ID":"586d100d-7063-44bf-9004-0265d19b5bc5","Type":"ContainerDied","Data":"1e7e62a1a7de229aa2397989c96264aca3e0ac58b70df7e2db2e5991560e47a6"}
Jul 15 19:16:05.863193 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:05.863163    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:06.489595 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:06.489567    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:06.489940 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:06.489684    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:16:06.489940 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:06.489715    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:06.489940 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:06.489764    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:16:06.697699 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:06.697667    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qm84p" event={"ID":"586d100d-7063-44bf-9004-0265d19b5bc5","Type":"ContainerStarted","Data":"797c76188634fc87d7c8bb844d8202f3bd5ba2e2c878933a849e5b17fc21907d"}
Jul 15 19:16:06.719907 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:06.719858    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qm84p" podStartSLOduration=9.361627479 podStartE2EDuration="38.719844309s" totalImagesPullingTime="29.35821683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:06.717769181 +0000 UTC m=+38.787628578" watchObservedRunningTime="2026-07-15 19:16:06.719844309 +0000 UTC m=+38.789703705"
Jul 15 19:16:07.489935 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:07.489907    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:07.490295 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:07.489991    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:16:08.489891 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:08.489864    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:08.490083 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:08.489943    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:16:08.490083 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:08.490036    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:08.490350 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:08.490144    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:16:09.489714 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:09.489679    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:09.489882 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:09.489791    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:16:10.489630 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:10.489594    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:10.490030 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:10.489712    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:16:10.490030 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:10.489765    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:10.490030 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:10.489864    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:16:11.489028 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:11.488997    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:11.489187 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:11.489107    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:16:12.489557 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:12.489522    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:12.489907 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:12.489566    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:12.489907 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:12.489648    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:16:12.489907 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:12.489771    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:16:13.489326 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:13.489298    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:13.489577 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:13.489386    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:16:14.489618 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:14.489584    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:14.489991 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:14.489687    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:16:14.489991 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:14.489741    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:14.489991 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:14.489826    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:16:15.489546 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:15.489518    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:15.489696 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:15.489604    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:16:16.318347 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:16.318317    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:16.368609 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:16.368576    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:16.489438 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:16.489402    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:16.489610 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:16.489557    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:16:16.489610 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:16.489594    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:16.489686 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:16.489668    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:16:17.489737 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:17.489701    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:17.490092 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:17.489809    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:16:17.590850 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:17.590820    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:17.752641 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:17.752568    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:17.841543 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:17.841511    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:17.977902 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:17.977870    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:18.076404 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:18.076328    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:18.140924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:18.140893    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:18.490044 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:18.490015    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:18.490467 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:18.490115    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:16:18.490467 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:18.490193    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:18.490467 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:18.490277    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:16:19.058267 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:19.058240    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sk27q"]
Jul 15 19:16:19.058425 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:19.058348    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:19.058506 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:19.058439    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:16:19.061504 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:19.061477    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7srhg"]
Jul 15 19:16:19.061632 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:19.061592    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:19.061722 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:19.061692    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:16:19.062286 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:19.062265    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-djwqm"]
Jul 15 19:16:19.062373 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:19.062334    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:19.062424 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:19.062396    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:16:20.489677 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:20.489648    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:20.489677 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:20.489679    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:20.490415 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:20.489648    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:20.490415 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:20.489770    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:16:20.490415 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:20.489821    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:16:20.490415 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:20.489908    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:16:21.907363 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:21.907195    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:21.961603 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:21.961574    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:22.489308 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:22.489280    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:22.489488 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:22.489280    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:22.489488 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:22.489382    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7srhg" podUID="955ee6c7-972b-4215-a5dc-ccdcb08f8da2"
Jul 15 19:16:22.489488 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:22.489285    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:22.489488 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:22.489445    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djwqm" podUID="995ebd78-54f8-4755-8261-bdca88870130"
Jul 15 19:16:22.489705 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:22.489553    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sk27q" podUID="cb2e4028-856d-4dba-bf82-1d056e6d2da2"
Jul 15 19:16:22.890655 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:22.890580    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:23.699945 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.699873    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-114.ec2.internal" event="NodeReady"
Jul 15 19:16:23.700292 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.699991    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 19:16:23.739376 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.739341    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-96b457b94-jz987"]
Jul 15 19:16:23.741381 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.741365    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.746171 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.746147    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 15 19:16:23.746479 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.746443    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 15 19:16:23.746694 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.746680    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 15 19:16:23.747500 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.747478    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tlkgd\""
Jul 15 19:16:23.753576 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.753558    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 15 19:16:23.761655 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.761638    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pbzx7"]
Jul 15 19:16:23.763210 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.763196    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:23.765381 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.765351    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hjh4v"]
Jul 15 19:16:23.766086 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.766062    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8qrjh\""
Jul 15 19:16:23.766180 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.766092    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 19:16:23.766894 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.766877    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 19:16:23.769518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.767530    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-96b457b94-jz987"]
Jul 15 19:16:23.769518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.767646    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hjh4v"
Jul 15 19:16:23.770916 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.770856    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 19:16:23.771021 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.770957    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-swcsq\""
Jul 15 19:16:23.771021 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.771002    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 19:16:23.772698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.772681    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 19:16:23.780316 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.780297    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmvdr\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-kube-api-access-kmvdr\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.780385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.780327    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-bound-sa-token\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.780434 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.780414    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-registry-tls\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.780501 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.780474    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9356193f-ee01-4c43-8ea1-7550f4b082af-ca-trust-extracted\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.780599 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.780504    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9356193f-ee01-4c43-8ea1-7550f4b082af-trusted-ca\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.780599 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.780530    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9356193f-ee01-4c43-8ea1-7550f4b082af-registry-certificates\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.780680 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.780600    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9356193f-ee01-4c43-8ea1-7550f4b082af-image-registry-private-configuration\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.780680 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.780657    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9356193f-ee01-4c43-8ea1-7550f4b082af-installation-pull-secrets\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.781437 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.781418    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pbzx7"]
Jul 15 19:16:23.784387 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.784317    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hjh4v"]
Jul 15 19:16:23.881082 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881042    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-registry-tls\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.881082 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881077    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9356193f-ee01-4c43-8ea1-7550f4b082af-ca-trust-extracted\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.881296 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881097    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9356193f-ee01-4c43-8ea1-7550f4b082af-trusted-ca\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.881296 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881116    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4tvz\" (UniqueName: \"kubernetes.io/projected/eb78d556-5ce2-4076-835c-68be95f7b4e6-kube-api-access-d4tvz\") pod \"dns-default-pbzx7\" (UID: \"eb78d556-5ce2-4076-835c-68be95f7b4e6\") " pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:23.881296 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881202    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9356193f-ee01-4c43-8ea1-7550f4b082af-registry-certificates\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.881296 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881220    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb78d556-5ce2-4076-835c-68be95f7b4e6-config-volume\") pod \"dns-default-pbzx7\" (UID: \"eb78d556-5ce2-4076-835c-68be95f7b4e6\") " pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:23.881296 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881249    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9356193f-ee01-4c43-8ea1-7550f4b082af-image-registry-private-configuration\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.881296 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881276    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9356193f-ee01-4c43-8ea1-7550f4b082af-installation-pull-secrets\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.881607 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881303    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb78d556-5ce2-4076-835c-68be95f7b4e6-tmp-dir\") pod \"dns-default-pbzx7\" (UID: \"eb78d556-5ce2-4076-835c-68be95f7b4e6\") " pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:23.881607 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881324    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwcrz\" (UniqueName: \"kubernetes.io/projected/b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5-kube-api-access-nwcrz\") pod \"ingress-canary-hjh4v\" (UID: \"b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5\") " pod="openshift-ingress-canary/ingress-canary-hjh4v"
Jul 15 19:16:23.881607 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881356    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmvdr\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-kube-api-access-kmvdr\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.881607 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881384    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-bound-sa-token\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.881607 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881422    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb78d556-5ce2-4076-835c-68be95f7b4e6-metrics-tls\") pod \"dns-default-pbzx7\" (UID: \"eb78d556-5ce2-4076-835c-68be95f7b4e6\") " pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:23.881607 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881486    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5-cert\") pod \"ingress-canary-hjh4v\" (UID: \"b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5\") " pod="openshift-ingress-canary/ingress-canary-hjh4v"
Jul 15 19:16:23.881607 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.881551    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9356193f-ee01-4c43-8ea1-7550f4b082af-ca-trust-extracted\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.882076 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.882050    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9356193f-ee01-4c43-8ea1-7550f4b082af-trusted-ca\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.882202 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.882095    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9356193f-ee01-4c43-8ea1-7550f4b082af-registry-certificates\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.882543 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.882522    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:23.885178 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.885157    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9356193f-ee01-4c43-8ea1-7550f4b082af-installation-pull-secrets\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.885178 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.885173    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9356193f-ee01-4c43-8ea1-7550f4b082af-image-registry-private-configuration\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.885326 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.885269    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-registry-tls\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.888944 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.888920    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-bound-sa-token\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.889316 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.889296    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmvdr\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-kube-api-access-kmvdr\") pod \"image-registry-96b457b94-jz987\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") " pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:23.982334 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.982250    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb78d556-5ce2-4076-835c-68be95f7b4e6-config-volume\") pod \"dns-default-pbzx7\" (UID: \"eb78d556-5ce2-4076-835c-68be95f7b4e6\") " pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:23.982334 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.982296    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb78d556-5ce2-4076-835c-68be95f7b4e6-tmp-dir\") pod \"dns-default-pbzx7\" (UID: \"eb78d556-5ce2-4076-835c-68be95f7b4e6\") " pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:23.982334 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.982313    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwcrz\" (UniqueName: \"kubernetes.io/projected/b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5-kube-api-access-nwcrz\") pod \"ingress-canary-hjh4v\" (UID: \"b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5\") " pod="openshift-ingress-canary/ingress-canary-hjh4v"
Jul 15 19:16:23.982579 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.982345    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb78d556-5ce2-4076-835c-68be95f7b4e6-metrics-tls\") pod \"dns-default-pbzx7\" (UID: \"eb78d556-5ce2-4076-835c-68be95f7b4e6\") " pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:23.982579 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.982358    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5-cert\") pod \"ingress-canary-hjh4v\" (UID: \"b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5\") " pod="openshift-ingress-canary/ingress-canary-hjh4v"
Jul 15 19:16:23.982579 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.982382    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4tvz\" (UniqueName: \"kubernetes.io/projected/eb78d556-5ce2-4076-835c-68be95f7b4e6-kube-api-access-d4tvz\") pod \"dns-default-pbzx7\" (UID: \"eb78d556-5ce2-4076-835c-68be95f7b4e6\") " pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:23.982704 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.982682    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb78d556-5ce2-4076-835c-68be95f7b4e6-tmp-dir\") pod \"dns-default-pbzx7\" (UID: \"eb78d556-5ce2-4076-835c-68be95f7b4e6\") " pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:23.982875 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.982844    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb78d556-5ce2-4076-835c-68be95f7b4e6-config-volume\") pod \"dns-default-pbzx7\" (UID: \"eb78d556-5ce2-4076-835c-68be95f7b4e6\") " pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:23.984610 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.984591    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb78d556-5ce2-4076-835c-68be95f7b4e6-metrics-tls\") pod \"dns-default-pbzx7\" (UID: \"eb78d556-5ce2-4076-835c-68be95f7b4e6\") " pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:23.984716 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.984695    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5-cert\") pod \"ingress-canary-hjh4v\" (UID: \"b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5\") " pod="openshift-ingress-canary/ingress-canary-hjh4v"
Jul 15 19:16:23.990109 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.990086    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwcrz\" (UniqueName: \"kubernetes.io/projected/b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5-kube-api-access-nwcrz\") pod \"ingress-canary-hjh4v\" (UID: \"b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5\") " pod="openshift-ingress-canary/ingress-canary-hjh4v"
Jul 15 19:16:23.990224 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:23.990197    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4tvz\" (UniqueName: \"kubernetes.io/projected/eb78d556-5ce2-4076-835c-68be95f7b4e6-kube-api-access-d4tvz\") pod \"dns-default-pbzx7\" (UID: \"eb78d556-5ce2-4076-835c-68be95f7b4e6\") " pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:24.050055 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.050024    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:24.076032 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.076005    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:24.081122 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.081101    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hjh4v"
Jul 15 19:16:24.204347 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.202870    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-96b457b94-jz987"]
Jul 15 19:16:24.207664 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:24.207621    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9356193f_ee01_4c43_8ea1_7550f4b082af.slice/crio-5424fec57b20442000cbfc321a635730567c519b2508218d36cdd021f99e8bd8 WatchSource:0}: Error finding container 5424fec57b20442000cbfc321a635730567c519b2508218d36cdd021f99e8bd8: Status 404 returned error can't find the container with id 5424fec57b20442000cbfc321a635730567c519b2508218d36cdd021f99e8bd8
Jul 15 19:16:24.221632 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.221559    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pbzx7"]
Jul 15 19:16:24.222340 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.222313    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:24.224432 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:24.224406    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb78d556_5ce2_4076_835c_68be95f7b4e6.slice/crio-d2992b1196904c501a211e9a640aabb3ccfc114270acf7060da86efb50cafd7c WatchSource:0}: Error finding container d2992b1196904c501a211e9a640aabb3ccfc114270acf7060da86efb50cafd7c: Status 404 returned error can't find the container with id d2992b1196904c501a211e9a640aabb3ccfc114270acf7060da86efb50cafd7c
Jul 15 19:16:24.238270 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.238237    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hjh4v"]
Jul 15 19:16:24.241130 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:24.241098    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7cc6da0_11f7_4f0a_b5fe_2f5c90fce9c5.slice/crio-37cc2d7507befe305f6bd0532c5098177d04501c91c74c14cd60fee869cc7a62 WatchSource:0}: Error finding container 37cc2d7507befe305f6bd0532c5098177d04501c91c74c14cd60fee869cc7a62: Status 404 returned error can't find the container with id 37cc2d7507befe305f6bd0532c5098177d04501c91c74c14cd60fee869cc7a62
Jul 15 19:16:24.317678 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.317652    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:24.489577 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.489499    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:24.489577 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.489524    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:24.489577 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.489507    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:24.492655 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.492626    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 19:16:24.492787 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.492673    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w6x7t\""
Jul 15 19:16:24.492787 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.492704    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 19:16:24.492787 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.492626    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 19:16:24.492787 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.492764    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pzvzj\""
Jul 15 19:16:24.493588 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.493572    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 19:16:24.736445 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.736407    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hjh4v" event={"ID":"b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5","Type":"ContainerStarted","Data":"37cc2d7507befe305f6bd0532c5098177d04501c91c74c14cd60fee869cc7a62"}
Jul 15 19:16:24.737812 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.737779    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pbzx7" event={"ID":"eb78d556-5ce2-4076-835c-68be95f7b4e6","Type":"ContainerStarted","Data":"d2992b1196904c501a211e9a640aabb3ccfc114270acf7060da86efb50cafd7c"}
Jul 15 19:16:24.739363 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.739339    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-96b457b94-jz987" event={"ID":"9356193f-ee01-4c43-8ea1-7550f4b082af","Type":"ContainerStarted","Data":"bc3061d5729ebf86840a71a59e999c6afd965ae69d9bff1b8ca594f74cdd4d9e"}
Jul 15 19:16:24.739481 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.739379    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-96b457b94-jz987" event={"ID":"9356193f-ee01-4c43-8ea1-7550f4b082af","Type":"ContainerStarted","Data":"5424fec57b20442000cbfc321a635730567c519b2508218d36cdd021f99e8bd8"}
Jul 15 19:16:24.739481 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.739472    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:24.760931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:24.760859    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-96b457b94-jz987" podStartSLOduration=76.760844602 podStartE2EDuration="1m16.760844602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:24.75997421 +0000 UTC m=+56.829833607" watchObservedRunningTime="2026-07-15 19:16:24.760844602 +0000 UTC m=+56.830703977"
Jul 15 19:16:26.746163 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:26.746131    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pbzx7" event={"ID":"eb78d556-5ce2-4076-835c-68be95f7b4e6","Type":"ContainerStarted","Data":"c945c7ae875e566acf5b4ca2033f6e17834b99716345b616c99f3d99702e8e90"}
Jul 15 19:16:26.746511 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:26.746183    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pbzx7" event={"ID":"eb78d556-5ce2-4076-835c-68be95f7b4e6","Type":"ContainerStarted","Data":"0f0e3a1e4a350a877a8e1f51aae9f6186e2cb11d63e25e2a232c1a60fe477af1"}
Jul 15 19:16:26.746511 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:26.746272    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:26.762923 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:26.762889    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pbzx7" podStartSLOduration=2.072310913 podStartE2EDuration="3.762876399s" totalImagesPullingTime="1.690565486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:26.761543058 +0000 UTC m=+58.831402455" watchObservedRunningTime="2026-07-15 19:16:26.762876399 +0000 UTC m=+58.832735795"
Jul 15 19:16:27.749839 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:27.749796    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hjh4v" event={"ID":"b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5","Type":"ContainerStarted","Data":"42b164124f9e344f52b828ee7101604df3966173aa4852fc0e76ed766ba2560d"}
Jul 15 19:16:27.768075 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:27.768031    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hjh4v" podStartSLOduration=2.255415209 podStartE2EDuration="4.76801614s" totalImagesPullingTime="2.512600931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:27.767420784 +0000 UTC m=+59.837280187" watchObservedRunningTime="2026-07-15 19:16:27.76801614 +0000 UTC m=+59.837875537"
Jul 15 19:16:28.679789 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:28.679764    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6vs2"
Jul 15 19:16:28.750139 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:28.750088    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:28.906327 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:28.906246    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x"]
Jul 15 19:16:28.906962 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:28.906937    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:28.924868 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:28.924849    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x"]
Jul 15 19:16:28.924965 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:28.924932    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x"
Jul 15 19:16:28.927259 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:28.927240    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 15 19:16:28.927675 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:28.927661    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xfm8x\""
Jul 15 19:16:28.981991 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:28.981969    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:29.018634 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.018605    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/da7a9830-a487-409e-9f40-0ae3607587b9-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-nfg2x\" (UID: \"da7a9830-a487-409e-9f40-0ae3607587b9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x"
Jul 15 19:16:29.065752 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.065710    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:29.119977 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.119948    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/da7a9830-a487-409e-9f40-0ae3607587b9-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-nfg2x\" (UID: \"da7a9830-a487-409e-9f40-0ae3607587b9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x"
Jul 15 19:16:29.120111 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:29.120095    2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 15 19:16:29.120175 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:29.120166    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da7a9830-a487-409e-9f40-0ae3607587b9-tls-certificates podName:da7a9830-a487-409e-9f40-0ae3607587b9 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:29.620147071 +0000 UTC m=+61.690006451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/da7a9830-a487-409e-9f40-0ae3607587b9-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-nfg2x" (UID: "da7a9830-a487-409e-9f40-0ae3607587b9") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 15 19:16:29.174804 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.174742    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-9z7pn"]
Jul 15 19:16:29.177636 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.177613    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:29.193663 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.193643    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-9z7pn"]
Jul 15 19:16:29.193739 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.193718    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-9z7pn"
Jul 15 19:16:29.196430 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.196413    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 19:16:29.196532 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.196437    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 19:16:29.196532 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.196417    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-2ss2c\""
Jul 15 19:16:29.220744 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.220720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d4ts\" (UniqueName: \"kubernetes.io/projected/b8521d92-c8db-4396-9a67-ebb0c3acb319-kube-api-access-5d4ts\") pod \"downloads-68c9cbb497-9z7pn\" (UID: \"b8521d92-c8db-4396-9a67-ebb0c3acb319\") " pod="openshift-console/downloads-68c9cbb497-9z7pn"
Jul 15 19:16:29.244712 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.244691    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:29.305767 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.305744    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:29.321139 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.321119    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d4ts\" (UniqueName: \"kubernetes.io/projected/b8521d92-c8db-4396-9a67-ebb0c3acb319-kube-api-access-5d4ts\") pod \"downloads-68c9cbb497-9z7pn\" (UID: \"b8521d92-c8db-4396-9a67-ebb0c3acb319\") " pod="openshift-console/downloads-68c9cbb497-9z7pn"
Jul 15 19:16:29.329749 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.329724    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d4ts\" (UniqueName: \"kubernetes.io/projected/b8521d92-c8db-4396-9a67-ebb0c3acb319-kube-api-access-5d4ts\") pod \"downloads-68c9cbb497-9z7pn\" (UID: \"b8521d92-c8db-4396-9a67-ebb0c3acb319\") " pod="openshift-console/downloads-68c9cbb497-9z7pn"
Jul 15 19:16:29.370589 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.370562    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:29.467588 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.467518    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:29.502738 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.502712    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-9z7pn"
Jul 15 19:16:29.623733 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.623705    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/da7a9830-a487-409e-9f40-0ae3607587b9-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-nfg2x\" (UID: \"da7a9830-a487-409e-9f40-0ae3607587b9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x"
Jul 15 19:16:29.625860 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.625838    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/da7a9830-a487-409e-9f40-0ae3607587b9-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-nfg2x\" (UID: \"da7a9830-a487-409e-9f40-0ae3607587b9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x"
Jul 15 19:16:29.631315 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.631293    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-9z7pn"]
Jul 15 19:16:29.633879 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:29.633856    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8521d92_c8db_4396_9a67_ebb0c3acb319.slice/crio-7cd96258a7f64ba7ae6bd7e01a3b6f8e02b41761b07f1377668a8bcbc69c04ae WatchSource:0}: Error finding container 7cd96258a7f64ba7ae6bd7e01a3b6f8e02b41761b07f1377668a8bcbc69c04ae: Status 404 returned error can't find the container with id 7cd96258a7f64ba7ae6bd7e01a3b6f8e02b41761b07f1377668a8bcbc69c04ae
Jul 15 19:16:29.754765 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.754728    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-9z7pn" event={"ID":"b8521d92-c8db-4396-9a67-ebb0c3acb319","Type":"ContainerStarted","Data":"7cd96258a7f64ba7ae6bd7e01a3b6f8e02b41761b07f1377668a8bcbc69c04ae"}
Jul 15 19:16:29.763264 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.763237    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hnzkb"]
Jul 15 19:16:29.788596 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.788565    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:29.793401 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.793381    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hnzkb"]
Jul 15 19:16:29.793526 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.793510    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.795825 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.795809    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 19:16:29.796119 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.796100    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 19:16:29.796235 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.796176    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bl2wc\""
Jul 15 19:16:29.796235 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.796181    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 19:16:29.796350 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.796333    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 19:16:29.825960 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.825932    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c42c996f-f3ac-450e-bac3-79c20674a579-crio-socket\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.826076 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.825989    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c42c996f-f3ac-450e-bac3-79c20674a579-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.826076 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.826016    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvlhb\" (UniqueName: \"kubernetes.io/projected/c42c996f-f3ac-450e-bac3-79c20674a579-kube-api-access-dvlhb\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.826076 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.826043    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c42c996f-f3ac-450e-bac3-79c20674a579-data-volume\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.826178 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.826149    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c42c996f-f3ac-450e-bac3-79c20674a579-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.832521 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.832502    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x"
Jul 15 19:16:29.852719 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.852700    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:29.914255 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.908029    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:29.914255 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.911665    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-96b457b94-jz987_9356193f-ee01-4c43-8ea1-7550f4b082af/registry/0.log"
Jul 15 19:16:29.927196 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.927168    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c42c996f-f3ac-450e-bac3-79c20674a579-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.927341 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.927216    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c42c996f-f3ac-450e-bac3-79c20674a579-crio-socket\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.927341 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.927255    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c42c996f-f3ac-450e-bac3-79c20674a579-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.927341 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:29.927279    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:16:29.927341 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.927280    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvlhb\" (UniqueName: \"kubernetes.io/projected/c42c996f-f3ac-450e-bac3-79c20674a579-kube-api-access-dvlhb\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.927556 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:29.927346    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c42c996f-f3ac-450e-bac3-79c20674a579-kube-rbac-proxy-cm podName:c42c996f-f3ac-450e-bac3-79c20674a579 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:30.427327805 +0000 UTC m=+62.497187197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c42c996f-f3ac-450e-bac3-79c20674a579-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hnzkb" (UID: "c42c996f-f3ac-450e-bac3-79c20674a579") : configmap "kube-rbac-proxy" not found
Jul 15 19:16:29.927556 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.927376    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c42c996f-f3ac-450e-bac3-79c20674a579-data-volume\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.927556 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.927378    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c42c996f-f3ac-450e-bac3-79c20674a579-crio-socket\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.927556 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:29.927405    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:16:29.927556 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:29.927472    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c42c996f-f3ac-450e-bac3-79c20674a579-insights-runtime-extractor-tls podName:c42c996f-f3ac-450e-bac3-79c20674a579 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:30.427438174 +0000 UTC m=+62.497297549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c42c996f-f3ac-450e-bac3-79c20674a579-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hnzkb" (UID: "c42c996f-f3ac-450e-bac3-79c20674a579") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:16:29.927757 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.927671    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c42c996f-f3ac-450e-bac3-79c20674a579-data-volume\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.935165 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.935147    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvlhb\" (UniqueName: \"kubernetes.io/projected/c42c996f-f3ac-450e-bac3-79c20674a579-kube-api-access-dvlhb\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:29.972294 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:29.972268    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x"]
Jul 15 19:16:29.975748 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:29.975725    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda7a9830_a487_409e_9f40_0ae3607587b9.slice/crio-221b1ab507a89241165cf68dade8f42382cb2addd2a6e2c34cf8bfbb4cfd9160 WatchSource:0}: Error finding container 221b1ab507a89241165cf68dade8f42382cb2addd2a6e2c34cf8bfbb4cfd9160: Status 404 returned error can't find the container with id 221b1ab507a89241165cf68dade8f42382cb2addd2a6e2c34cf8bfbb4cfd9160
Jul 15 19:16:30.266222 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:30.266192    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:30.311027 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:30.311004    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lgzhf_1f63fb98-ad9f-4986-af0f-714ae207a152/node-ca/0.log"
Jul 15 19:16:30.431518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:30.431484    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c42c996f-f3ac-450e-bac3-79c20674a579-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:30.431704 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:30.431589    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c42c996f-f3ac-450e-bac3-79c20674a579-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:30.431704 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:30.431647    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 19:16:30.431704 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:30.431695    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 19:16:30.431896 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:30.431719    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c42c996f-f3ac-450e-bac3-79c20674a579-insights-runtime-extractor-tls podName:c42c996f-f3ac-450e-bac3-79c20674a579 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:31.431701931 +0000 UTC m=+63.501561311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c42c996f-f3ac-450e-bac3-79c20674a579-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hnzkb" (UID: "c42c996f-f3ac-450e-bac3-79c20674a579") : secret "insights-runtime-extractor-tls" not found
Jul 15 19:16:30.431896 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:30.431757    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c42c996f-f3ac-450e-bac3-79c20674a579-kube-rbac-proxy-cm podName:c42c996f-f3ac-450e-bac3-79c20674a579 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:31.431740981 +0000 UTC m=+63.501600373 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c42c996f-f3ac-450e-bac3-79c20674a579-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hnzkb" (UID: "c42c996f-f3ac-450e-bac3-79c20674a579") : configmap "kube-rbac-proxy" not found
Jul 15 19:16:30.759344 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:30.759302    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x" event={"ID":"da7a9830-a487-409e-9f40-0ae3607587b9","Type":"ContainerStarted","Data":"221b1ab507a89241165cf68dade8f42382cb2addd2a6e2c34cf8bfbb4cfd9160"}
Jul 15 19:16:30.786937 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:30.786906    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:30.888568 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:30.888534    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:30.911524 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:30.911487    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hjh4v_b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5/serve-healthcheck-canary/0.log"
Jul 15 19:16:30.988656 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:30.988625    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:31.157291 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.157218    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:31.314128 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.314106    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:31.362786 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.362745    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:31.440712 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.440619    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c42c996f-f3ac-450e-bac3-79c20674a579-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:31.440874 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.440803    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c42c996f-f3ac-450e-bac3-79c20674a579-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:31.441268 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.441224    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c42c996f-f3ac-450e-bac3-79c20674a579-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:31.443085 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.443061    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c42c996f-f3ac-450e-bac3-79c20674a579-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnzkb\" (UID: \"c42c996f-f3ac-450e-bac3-79c20674a579\") " pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:31.602564 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.602528    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hnzkb"
Jul 15 19:16:31.737034 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.737001    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hnzkb"]
Jul 15 19:16:31.740712 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:31.740682    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42c996f_f3ac_450e_bac3_79c20674a579.slice/crio-edf5a26c8dba6f063831052af05d2cd699bd8c2b8d46f4e8495b6d4ed95b5522 WatchSource:0}: Error finding container edf5a26c8dba6f063831052af05d2cd699bd8c2b8d46f4e8495b6d4ed95b5522: Status 404 returned error can't find the container with id edf5a26c8dba6f063831052af05d2cd699bd8c2b8d46f4e8495b6d4ed95b5522
Jul 15 19:16:31.764597 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.764561    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x" event={"ID":"da7a9830-a487-409e-9f40-0ae3607587b9","Type":"ContainerStarted","Data":"8a779c86a083a33edccab70a2fbefa9c5736f698d3ecc8af3dada1555b4a2ab4"}
Jul 15 19:16:31.765038 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.764725    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x"
Jul 15 19:16:31.765884 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.765858    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnzkb" event={"ID":"c42c996f-f3ac-450e-bac3-79c20674a579","Type":"ContainerStarted","Data":"edf5a26c8dba6f063831052af05d2cd699bd8c2b8d46f4e8495b6d4ed95b5522"}
Jul 15 19:16:31.769946 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.769921    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x"
Jul 15 19:16:31.779984 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.779942    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-nfg2x" podStartSLOduration=2.509725731 podStartE2EDuration="3.77992786s" totalImagesPullingTime="1.270202129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:31.778816375 +0000 UTC m=+63.848675773" watchObservedRunningTime="2026-07-15 19:16:31.77992786 +0000 UTC m=+63.849787257"
Jul 15 19:16:31.817107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.817078    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:31.961339 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.961305    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-nfqd2"]
Jul 15 19:16:31.968909 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.968886    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:31.971598 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.971551    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 19:16:31.971749 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.971709    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-szl6s\""
Jul 15 19:16:31.971749 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.971726    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 19:16:31.971890 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.971790    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 19:16:31.971890 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.971710    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 15 19:16:31.972045 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.972007    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 15 19:16:31.972785 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:31.972767    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-nfqd2"]
Jul 15 19:16:32.045821 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.045736    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a851c47d-8ee9-414c-bded-b74bb2105be5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-nfqd2\" (UID: \"a851c47d-8ee9-414c-bded-b74bb2105be5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:32.045821 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.045783    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a851c47d-8ee9-414c-bded-b74bb2105be5-metrics-client-ca\") pod \"prometheus-operator-64785b6675-nfqd2\" (UID: \"a851c47d-8ee9-414c-bded-b74bb2105be5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:32.046003 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.045821    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a851c47d-8ee9-414c-bded-b74bb2105be5-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-nfqd2\" (UID: \"a851c47d-8ee9-414c-bded-b74bb2105be5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:32.046003 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.045845    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sczvb\" (UniqueName: \"kubernetes.io/projected/a851c47d-8ee9-414c-bded-b74bb2105be5-kube-api-access-sczvb\") pod \"prometheus-operator-64785b6675-nfqd2\" (UID: \"a851c47d-8ee9-414c-bded-b74bb2105be5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:32.148477 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.147103    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a851c47d-8ee9-414c-bded-b74bb2105be5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-nfqd2\" (UID: \"a851c47d-8ee9-414c-bded-b74bb2105be5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:32.148477 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.147164    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a851c47d-8ee9-414c-bded-b74bb2105be5-metrics-client-ca\") pod \"prometheus-operator-64785b6675-nfqd2\" (UID: \"a851c47d-8ee9-414c-bded-b74bb2105be5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:32.148477 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.147211    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a851c47d-8ee9-414c-bded-b74bb2105be5-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-nfqd2\" (UID: \"a851c47d-8ee9-414c-bded-b74bb2105be5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:32.148477 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.147254    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sczvb\" (UniqueName: \"kubernetes.io/projected/a851c47d-8ee9-414c-bded-b74bb2105be5-kube-api-access-sczvb\") pod \"prometheus-operator-64785b6675-nfqd2\" (UID: \"a851c47d-8ee9-414c-bded-b74bb2105be5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:32.148477 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:32.148116    2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 15 19:16:32.148477 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:32.148203    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a851c47d-8ee9-414c-bded-b74bb2105be5-prometheus-operator-tls podName:a851c47d-8ee9-414c-bded-b74bb2105be5 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:32.64817876 +0000 UTC m=+64.718038157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/a851c47d-8ee9-414c-bded-b74bb2105be5-prometheus-operator-tls") pod "prometheus-operator-64785b6675-nfqd2" (UID: "a851c47d-8ee9-414c-bded-b74bb2105be5") : secret "prometheus-operator-tls" not found
Jul 15 19:16:32.149369 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.149168    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a851c47d-8ee9-414c-bded-b74bb2105be5-metrics-client-ca\") pod \"prometheus-operator-64785b6675-nfqd2\" (UID: \"a851c47d-8ee9-414c-bded-b74bb2105be5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:32.152092 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.152046    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a851c47d-8ee9-414c-bded-b74bb2105be5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-nfqd2\" (UID: \"a851c47d-8ee9-414c-bded-b74bb2105be5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:32.160101 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.160078    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sczvb\" (UniqueName: \"kubernetes.io/projected/a851c47d-8ee9-414c-bded-b74bb2105be5-kube-api-access-sczvb\") pod \"prometheus-operator-64785b6675-nfqd2\" (UID: \"a851c47d-8ee9-414c-bded-b74bb2105be5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:32.652018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.651573    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a851c47d-8ee9-414c-bded-b74bb2105be5-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-nfqd2\" (UID: \"a851c47d-8ee9-414c-bded-b74bb2105be5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:32.654890 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.654859    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a851c47d-8ee9-414c-bded-b74bb2105be5-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-nfqd2\" (UID: \"a851c47d-8ee9-414c-bded-b74bb2105be5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:32.879317 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:32.879283    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2"
Jul 15 19:16:33.017860 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:33.017794    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-nfqd2"]
Jul 15 19:16:33.021594 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:33.021568    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda851c47d_8ee9_414c_bded_b74bb2105be5.slice/crio-15490d3e5b6dc0b2013cbf6e3ff8044c2746dc6d59b2f89eded787ad4e9777c4 WatchSource:0}: Error finding container 15490d3e5b6dc0b2013cbf6e3ff8044c2746dc6d59b2f89eded787ad4e9777c4: Status 404 returned error can't find the container with id 15490d3e5b6dc0b2013cbf6e3ff8044c2746dc6d59b2f89eded787ad4e9777c4
Jul 15 19:16:33.339985 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:33.339920    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:33.400612 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:33.400574    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:33.776315 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:33.776281    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2" event={"ID":"a851c47d-8ee9-414c-bded-b74bb2105be5","Type":"ContainerStarted","Data":"15490d3e5b6dc0b2013cbf6e3ff8044c2746dc6d59b2f89eded787ad4e9777c4"}
Jul 15 19:16:34.165951 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.165856    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs\") pod \"network-metrics-daemon-sk27q\" (UID: \"cb2e4028-856d-4dba-bf82-1d056e6d2da2\") " pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:34.165951 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.165928    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:34.168743 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.168713    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 19:16:34.169628 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.169608    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 19:16:34.179340 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.179313    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/995ebd78-54f8-4755-8261-bdca88870130-original-pull-secret\") pod \"global-pull-secret-syncer-djwqm\" (UID: \"995ebd78-54f8-4755-8261-bdca88870130\") " pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:34.179479 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.179427    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb2e4028-856d-4dba-bf82-1d056e6d2da2-metrics-certs\") pod \"network-metrics-daemon-sk27q\" (UID: \"cb2e4028-856d-4dba-bf82-1d056e6d2da2\") " pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:34.266868 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.266831    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx25j\" (UniqueName: \"kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j\") pod \"network-check-target-7srhg\" (UID: \"955ee6c7-972b-4215-a5dc-ccdcb08f8da2\") " pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:34.269793 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.269769    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 19:16:34.280384 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.280357    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 19:16:34.293005 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.292980    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx25j\" (UniqueName: \"kubernetes.io/projected/955ee6c7-972b-4215-a5dc-ccdcb08f8da2-kube-api-access-qx25j\") pod \"network-check-target-7srhg\" (UID: \"955ee6c7-972b-4215-a5dc-ccdcb08f8da2\") " pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:34.399166 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.399128    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djwqm"
Jul 15 19:16:34.406964 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.406935    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w6x7t\""
Jul 15 19:16:34.411183 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.411163    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pzvzj\""
Jul 15 19:16:34.415268 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.415242    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sk27q"
Jul 15 19:16:34.419419 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:34.419400    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:36.752250 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.752219    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pbzx7"
Jul 15 19:16:36.910192 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.910158    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cf74b5f5b-8572m"]
Jul 15 19:16:36.926763 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.926738    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cf74b5f5b-8572m"]
Jul 15 19:16:36.926921 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.926856    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:36.930312 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.930285    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 15 19:16:36.930432 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.930327    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 15 19:16:36.930432 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.930418    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 15 19:16:36.930794 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.930650    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6hvzs\""
Jul 15 19:16:36.930794 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.930669    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 15 19:16:36.930918 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.930892    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 15 19:16:36.988809 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.988783    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-oauth-serving-cert\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:36.988947 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.988824    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-config\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:36.988947 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.988853    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-service-ca\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:36.989053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.988993    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-serving-cert\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:36.989053 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.989027    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-oauth-config\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:36.989146 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:36.989064    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78vwf\" (UniqueName: \"kubernetes.io/projected/518fd2e3-df4d-432f-a374-5b28ddfb77f0-kube-api-access-78vwf\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.057584 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.057510    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:37.089621 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.089578    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-serving-cert\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.089774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.089626    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-oauth-config\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.089774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.089682    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78vwf\" (UniqueName: \"kubernetes.io/projected/518fd2e3-df4d-432f-a374-5b28ddfb77f0-kube-api-access-78vwf\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.089774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.089754    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-oauth-serving-cert\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.089930 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.089798    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-config\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.089930 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.089823    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-service-ca\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.090577 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.090513    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-service-ca\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.090698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.090573    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-oauth-serving-cert\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.090698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.090611    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-config\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.092517 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.092492    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-oauth-config\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.092615 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.092543    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-serving-cert\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.098562 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.098380    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78vwf\" (UniqueName: \"kubernetes.io/projected/518fd2e3-df4d-432f-a374-5b28ddfb77f0-kube-api-access-78vwf\") pod \"console-5cf74b5f5b-8572m\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") " pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.159153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.159116    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:37.238369 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.238337    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:16:37.979742 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:37.979719    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7srhg"]
Jul 15 19:16:37.996634 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:37.996605    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod955ee6c7_972b_4215_a5dc_ccdcb08f8da2.slice/crio-454a45e7fb5852f30764982a8ede6604b597eeeae304b6c5dd0b2bcd7c45e685 WatchSource:0}: Error finding container 454a45e7fb5852f30764982a8ede6604b597eeeae304b6c5dd0b2bcd7c45e685: Status 404 returned error can't find the container with id 454a45e7fb5852f30764982a8ede6604b597eeeae304b6c5dd0b2bcd7c45e685
Jul 15 19:16:38.159120 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:38.159041    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:38.201692 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:38.201226    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-djwqm"]
Jul 15 19:16:38.204214 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:38.204187    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cf74b5f5b-8572m"]
Jul 15 19:16:38.204735 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:38.204715    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod995ebd78_54f8_4755_8261_bdca88870130.slice/crio-ec7d2e78ba873f451df9bda82c61a52ffbb2bb767ce711bc8c3a4293649cddbc WatchSource:0}: Error finding container ec7d2e78ba873f451df9bda82c61a52ffbb2bb767ce711bc8c3a4293649cddbc: Status 404 returned error can't find the container with id ec7d2e78ba873f451df9bda82c61a52ffbb2bb767ce711bc8c3a4293649cddbc
Jul 15 19:16:38.207786 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:38.207766    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod518fd2e3_df4d_432f_a374_5b28ddfb77f0.slice/crio-691a082080c2ec8fcf359a211601d93a8282a32c033dbbc26ea59a93cdd65b8c WatchSource:0}: Error finding container 691a082080c2ec8fcf359a211601d93a8282a32c033dbbc26ea59a93cdd65b8c: Status 404 returned error can't find the container with id 691a082080c2ec8fcf359a211601d93a8282a32c033dbbc26ea59a93cdd65b8c
Jul 15 19:16:38.211058 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:38.211034    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sk27q"]
Jul 15 19:16:38.214437 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:38.214409    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb2e4028_856d_4dba_bf82_1d056e6d2da2.slice/crio-f1435577cf027b57a1f5f92b33ef535e096f188c70229f8333788024b4284165 WatchSource:0}: Error finding container f1435577cf027b57a1f5f92b33ef535e096f188c70229f8333788024b4284165: Status 404 returned error can't find the container with id f1435577cf027b57a1f5f92b33ef535e096f188c70229f8333788024b4284165
Jul 15 19:16:38.806321 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:38.806182    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnzkb" event={"ID":"c42c996f-f3ac-450e-bac3-79c20674a579","Type":"ContainerStarted","Data":"69df174ebd54e7dccce82344f8bb2d853b8c999347f648c414f90dac04280d26"}
Jul 15 19:16:38.811740 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:38.811653    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2" event={"ID":"a851c47d-8ee9-414c-bded-b74bb2105be5","Type":"ContainerStarted","Data":"00605e8aeaccb3c3a05776ade424e2a62da4582c25e7acd76986bda00cff88e7"}
Jul 15 19:16:38.811740 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:38.811707    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2" event={"ID":"a851c47d-8ee9-414c-bded-b74bb2105be5","Type":"ContainerStarted","Data":"3416ee6a604986f0c75e8a0e063e586035305a575138356dbabd60ff005088e8"}
Jul 15 19:16:38.814783 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:38.814671    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf74b5f5b-8572m" event={"ID":"518fd2e3-df4d-432f-a374-5b28ddfb77f0","Type":"ContainerStarted","Data":"691a082080c2ec8fcf359a211601d93a8282a32c033dbbc26ea59a93cdd65b8c"}
Jul 15 19:16:38.818668 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:38.818589    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7srhg" event={"ID":"955ee6c7-972b-4215-a5dc-ccdcb08f8da2","Type":"ContainerStarted","Data":"454a45e7fb5852f30764982a8ede6604b597eeeae304b6c5dd0b2bcd7c45e685"}
Jul 15 19:16:38.823396 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:38.823332    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sk27q" event={"ID":"cb2e4028-856d-4dba-bf82-1d056e6d2da2","Type":"ContainerStarted","Data":"f1435577cf027b57a1f5f92b33ef535e096f188c70229f8333788024b4284165"}
Jul 15 19:16:38.826468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:38.826393    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-djwqm" event={"ID":"995ebd78-54f8-4755-8261-bdca88870130","Type":"ContainerStarted","Data":"ec7d2e78ba873f451df9bda82c61a52ffbb2bb767ce711bc8c3a4293649cddbc"}
Jul 15 19:16:40.041634 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.041600    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:40.307494 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.306771    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-nfqd2" podStartSLOduration=4.550290835 podStartE2EDuration="9.306741548s" totalImagesPullingTime="4.756450713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:38.831948304 +0000 UTC m=+70.901807702" watchObservedRunningTime="2026-07-15 19:16:40.306741548 +0000 UTC m=+72.376600946"
Jul 15 19:16:40.307494 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.306993    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"]
Jul 15 19:16:40.312209 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.311585    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"]
Jul 15 19:16:40.314534 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.313100    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:40.314534 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.314252    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.323106 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.319949    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ttvsd\""
Jul 15 19:16:40.323106 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.321607    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 15 19:16:40.323106 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.321830    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 15 19:16:40.323106 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.322094    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-d27h8\""
Jul 15 19:16:40.323106 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.322278    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 15 19:16:40.323106 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.322363    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 15 19:16:40.323106 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.322475    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 15 19:16:40.324927 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.324190    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rfhxr"]
Jul 15 19:16:40.332927 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.331999    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.334774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.334695    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"]
Jul 15 19:16:40.336913 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.335718    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"]
Jul 15 19:16:40.336913 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.335756    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q2bf5\""
Jul 15 19:16:40.336913 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.336103    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 19:16:40.336913 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.336319    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 19:16:40.336913 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.336559    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 19:16:40.346860 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.346544    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:40.423468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.422952    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.423468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423004    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-tls\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.423468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423047    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.423468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423087    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c88ecb0-35f5-46fb-8b64-31689b50e178-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.423468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423116    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.423468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423142    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b0da1ee-8857-46b5-a35d-630cb0817dfe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-hdm2p\" (UID: \"3b0da1ee-8857-46b5-a35d-630cb0817dfe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:40.423468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423170    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b0da1ee-8857-46b5-a35d-630cb0817dfe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-hdm2p\" (UID: \"3b0da1ee-8857-46b5-a35d-630cb0817dfe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:40.423468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423196    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/38246ac2-0ceb-4aef-a07b-50dbc742e79d-root\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.423468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423220    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4429\" (UniqueName: \"kubernetes.io/projected/38246ac2-0ceb-4aef-a07b-50dbc742e79d-kube-api-access-v4429\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.423468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423256    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b0da1ee-8857-46b5-a35d-630cb0817dfe-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-hdm2p\" (UID: \"3b0da1ee-8857-46b5-a35d-630cb0817dfe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:40.423468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423298    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.423468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423339    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-accelerators-collector-config\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.424157 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423375    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38246ac2-0ceb-4aef-a07b-50dbc742e79d-metrics-client-ca\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.424157 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423464    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-wtmp\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.424157 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423508    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9794g\" (UniqueName: \"kubernetes.io/projected/3b0da1ee-8857-46b5-a35d-630cb0817dfe-kube-api-access-9794g\") pod \"openshift-state-metrics-84bd68b896-hdm2p\" (UID: \"3b0da1ee-8857-46b5-a35d-630cb0817dfe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:40.424157 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423561    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1c88ecb0-35f5-46fb-8b64-31689b50e178-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.424157 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423584    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38246ac2-0ceb-4aef-a07b-50dbc742e79d-sys\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.424157 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423606    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-textfile\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.424157 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.423634    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swwpt\" (UniqueName: \"kubernetes.io/projected/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-api-access-swwpt\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.435181 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.435122    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:40.524941 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.524899    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-accelerators-collector-config\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.525153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.524959    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38246ac2-0ceb-4aef-a07b-50dbc742e79d-metrics-client-ca\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.525153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.524993    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-wtmp\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.525153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525028    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9794g\" (UniqueName: \"kubernetes.io/projected/3b0da1ee-8857-46b5-a35d-630cb0817dfe-kube-api-access-9794g\") pod \"openshift-state-metrics-84bd68b896-hdm2p\" (UID: \"3b0da1ee-8857-46b5-a35d-630cb0817dfe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:40.525153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525064    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1c88ecb0-35f5-46fb-8b64-31689b50e178-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.525153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525091    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38246ac2-0ceb-4aef-a07b-50dbc742e79d-sys\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.525153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525115    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-textfile\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.525153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525144    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swwpt\" (UniqueName: \"kubernetes.io/projected/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-api-access-swwpt\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.525524 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525175    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.525524 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525207    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-tls\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.525524 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525243    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.525524 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525288    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c88ecb0-35f5-46fb-8b64-31689b50e178-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.525524 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525315    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.525524 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525347    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b0da1ee-8857-46b5-a35d-630cb0817dfe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-hdm2p\" (UID: \"3b0da1ee-8857-46b5-a35d-630cb0817dfe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:40.525524 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525373    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b0da1ee-8857-46b5-a35d-630cb0817dfe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-hdm2p\" (UID: \"3b0da1ee-8857-46b5-a35d-630cb0817dfe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:40.525524 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525397    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/38246ac2-0ceb-4aef-a07b-50dbc742e79d-root\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.525524 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525423    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4429\" (UniqueName: \"kubernetes.io/projected/38246ac2-0ceb-4aef-a07b-50dbc742e79d-kube-api-access-v4429\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.525524 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525467    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b0da1ee-8857-46b5-a35d-630cb0817dfe-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-hdm2p\" (UID: \"3b0da1ee-8857-46b5-a35d-630cb0817dfe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:40.525524 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.525495    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.527071 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:40.526406    2560 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 15 19:16:40.527071 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:40.526499    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b0da1ee-8857-46b5-a35d-630cb0817dfe-openshift-state-metrics-tls podName:3b0da1ee-8857-46b5-a35d-630cb0817dfe nodeName:}" failed. No retries permitted until 2026-07-15 19:16:41.026476532 +0000 UTC m=+73.096335925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3b0da1ee-8857-46b5-a35d-630cb0817dfe-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-hdm2p" (UID: "3b0da1ee-8857-46b5-a35d-630cb0817dfe") : secret "openshift-state-metrics-tls" not found
Jul 15 19:16:40.527770 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.527721    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c88ecb0-35f5-46fb-8b64-31689b50e178-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.530506 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.528289    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.530506 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.528376    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/38246ac2-0ceb-4aef-a07b-50dbc742e79d-root\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.530506 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:40.528876    2560 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 15 19:16:40.530506 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:40.528933    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-state-metrics-tls podName:1c88ecb0-35f5-46fb-8b64-31689b50e178 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:41.028916129 +0000 UTC m=+73.098775508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-m5zgx" (UID: "1c88ecb0-35f5-46fb-8b64-31689b50e178") : secret "kube-state-metrics-tls" not found
Jul 15 19:16:40.530506 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.529428    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b0da1ee-8857-46b5-a35d-630cb0817dfe-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-hdm2p\" (UID: \"3b0da1ee-8857-46b5-a35d-630cb0817dfe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:40.530506 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.529526    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38246ac2-0ceb-4aef-a07b-50dbc742e79d-sys\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.530506 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.529525    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38246ac2-0ceb-4aef-a07b-50dbc742e79d-metrics-client-ca\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.530506 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.529813    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1c88ecb0-35f5-46fb-8b64-31689b50e178-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.530506 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.529913    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-accelerators-collector-config\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.530506 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.530195    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-textfile\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.530918 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.530881    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-wtmp\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.533616 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.533491    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b0da1ee-8857-46b5-a35d-630cb0817dfe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-hdm2p\" (UID: \"3b0da1ee-8857-46b5-a35d-630cb0817dfe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:40.538085 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.536131    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-tls\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.538085 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.537866    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:40.540063 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.540038    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38246ac2-0ceb-4aef-a07b-50dbc742e79d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.542153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.542131    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4429\" (UniqueName: \"kubernetes.io/projected/38246ac2-0ceb-4aef-a07b-50dbc742e79d-kube-api-access-v4429\") pod \"node-exporter-rfhxr\" (UID: \"38246ac2-0ceb-4aef-a07b-50dbc742e79d\") " pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.542298 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.542274    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.551507 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.551463    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9794g\" (UniqueName: \"kubernetes.io/projected/3b0da1ee-8857-46b5-a35d-630cb0817dfe-kube-api-access-9794g\") pod \"openshift-state-metrics-84bd68b896-hdm2p\" (UID: \"3b0da1ee-8857-46b5-a35d-630cb0817dfe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:40.553363 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.553318    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swwpt\" (UniqueName: \"kubernetes.io/projected/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-api-access-swwpt\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:40.663964 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.663647    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rfhxr"
Jul 15 19:16:40.687809 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.687734    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:40.780884 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.780837    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:40.888260 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.888225    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:40.981738 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:40.981661    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:41.030812 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.030751    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:41.031170 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.031028    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b0da1ee-8857-46b5-a35d-630cb0817dfe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-hdm2p\" (UID: \"3b0da1ee-8857-46b5-a35d-630cb0817dfe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:41.034366 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.034312    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1c88ecb0-35f5-46fb-8b64-31689b50e178-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-m5zgx\" (UID: \"1c88ecb0-35f5-46fb-8b64-31689b50e178\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:41.037937 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.037884    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3b0da1ee-8857-46b5-a35d-630cb0817dfe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-hdm2p\" (UID: \"3b0da1ee-8857-46b5-a35d-630cb0817dfe\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:41.047187 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.046128    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-96b457b94-jz987"]
Jul 15 19:16:41.054486 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.054465    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:16:41.091784 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.091692    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:41.186217 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.186183    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:41.243197 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.243163    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"
Jul 15 19:16:41.252289 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.252121    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"
Jul 15 19:16:41.329286 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.329192    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:41.431908 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.431870    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 19:16:41.435040 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.434981    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:41.439680 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.439175    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.441976 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.441811    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 15 19:16:41.445609 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.444246    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 15 19:16:41.445609 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.444750    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 15 19:16:41.445609 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.444879    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 15 19:16:41.445609 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.444983    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 15 19:16:41.445609 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.445030    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 15 19:16:41.445609 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.445162    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 15 19:16:41.445609 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.445261    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 15 19:16:41.445609 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.445340    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5wtsc\""
Jul 15 19:16:41.445609 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.445486    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 15 19:16:41.464580 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.464487    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 19:16:41.535412 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.535342    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-config-volume\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.535412 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.535389    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j4z4\" (UniqueName: \"kubernetes.io/projected/d76fb1a6-3241-475d-9059-0f347022da7c-kube-api-access-9j4z4\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.535636 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.535419    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.535636 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.535509    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.535636 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.535555    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-web-config\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.535636 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.535597    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d76fb1a6-3241-475d-9059-0f347022da7c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.535825 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.535681    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.535825 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.535724    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.535825 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.535755    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d76fb1a6-3241-475d-9059-0f347022da7c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.535825 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.535786    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d76fb1a6-3241-475d-9059-0f347022da7c-config-out\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.535825 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.535813    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d76fb1a6-3241-475d-9059-0f347022da7c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.535999 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.535869    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.535999 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.535912    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d76fb1a6-3241-475d-9059-0f347022da7c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.555337 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.555268    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:41.636909 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.636873    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d76fb1a6-3241-475d-9059-0f347022da7c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.637096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.636922    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.637096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.636965    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d76fb1a6-3241-475d-9059-0f347022da7c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.637096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.637003    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-config-volume\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.637096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.637028    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j4z4\" (UniqueName: \"kubernetes.io/projected/d76fb1a6-3241-475d-9059-0f347022da7c-kube-api-access-9j4z4\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.637096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.637058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.637096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.637088    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.637376 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.637114    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-web-config\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.637376 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.637138    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d76fb1a6-3241-475d-9059-0f347022da7c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.637376 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.637176    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.637376 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.637204    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.637376 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.637232    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d76fb1a6-3241-475d-9059-0f347022da7c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.637376 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.637263    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d76fb1a6-3241-475d-9059-0f347022da7c-config-out\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.638649 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.637698    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d76fb1a6-3241-475d-9059-0f347022da7c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.638649 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:41.638357    2560 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 15 19:16:41.638649 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:41.638426    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-main-tls podName:d76fb1a6-3241-475d-9059-0f347022da7c nodeName:}" failed. No retries permitted until 2026-07-15 19:16:42.138406269 +0000 UTC m=+74.208265662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c") : secret "alertmanager-main-tls" not found
Jul 15 19:16:41.638649 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.638530    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d76fb1a6-3241-475d-9059-0f347022da7c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.638944 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.638713    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d76fb1a6-3241-475d-9059-0f347022da7c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.642772 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.642691    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.643204 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.643180    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-config-volume\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.643634 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.643612    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d76fb1a6-3241-475d-9059-0f347022da7c-config-out\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.643937 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.643921    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.644087 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.644063    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.645437 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.645395    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-web-config\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.645862 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.645833    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d76fb1a6-3241-475d-9059-0f347022da7c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.646266 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.646233    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.648726 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.648683    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j4z4\" (UniqueName: \"kubernetes.io/projected/d76fb1a6-3241-475d-9059-0f347022da7c-kube-api-access-9j4z4\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:41.652567 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.652234    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:41.775635 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.775553    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:41.891769 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.890463    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8c7fdb7b8-nktpj"]
Jul 15 19:16:41.895366 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.894844    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:41.906482 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.905472    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 15 19:16:41.906756 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.906732    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8c7fdb7b8-nktpj"]
Jul 15 19:16:41.908308 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:41.908282    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:42.041321 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.041282    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-config\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.041513 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.041348    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-trusted-ca-bundle\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.041513 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.041401    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-oauth-config\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.041513 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.041445    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-service-ca\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.041513 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.041487    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-serving-cert\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.041736 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.041518    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-oauth-serving-cert\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.041736 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.041544    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbh5x\" (UniqueName: \"kubernetes.io/projected/1ac8b408-ead4-4902-b9f5-a665c2b35656-kube-api-access-zbh5x\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.142896 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.142812    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-oauth-config\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.142896 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.142885    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-service-ca\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.143385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.142913    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-serving-cert\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.143385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.142946    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-oauth-serving-cert\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.143385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.142975    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbh5x\" (UniqueName: \"kubernetes.io/projected/1ac8b408-ead4-4902-b9f5-a665c2b35656-kube-api-access-zbh5x\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.143385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.143028    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-config\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.143385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.143062    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:42.143385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.143086    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-trusted-ca-bundle\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.144282 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.144097    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-trusted-ca-bundle\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.147422 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.147370    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-oauth-config\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.152550 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.147918    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-config\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.152550 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.147926    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-oauth-serving-cert\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.152550 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.148843    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-service-ca\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.154262 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.153974    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:42.156618 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.156572    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-serving-cert\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.157663 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.157638    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbh5x\" (UniqueName: \"kubernetes.io/projected/1ac8b408-ead4-4902-b9f5-a665c2b35656-kube-api-access-zbh5x\") pod \"console-8c7fdb7b8-nktpj\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") " pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.208368 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.208335    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:16:42.353745 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.353706    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:16:42.409484 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.409393    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-775c58594c-dv5vh"]
Jul 15 19:16:42.413809 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.413781    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.416633 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.416610    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 15 19:16:42.416633 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.416629    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 15 19:16:42.416772 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.416659    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 15 19:16:42.416772 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.416611    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9xlmv\""
Jul 15 19:16:42.416881 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.416866    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 15 19:16:42.416993 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.416976    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e7tpcv61kvb0u\""
Jul 15 19:16:42.417070 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.417055    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 15 19:16:42.430615 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.430595    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-775c58594c-dv5vh"]
Jul 15 19:16:42.546262 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.546229    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.546428 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.546268    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.546428 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.546335    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.546428 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.546372    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-metrics-client-ca\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.546428 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.546399    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.546653 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.546427    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-tls\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.546653 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.546485    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-grpc-tls\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.546653 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.546518    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zwt2\" (UniqueName: \"kubernetes.io/projected/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-kube-api-access-9zwt2\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.647544 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.647501    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zwt2\" (UniqueName: \"kubernetes.io/projected/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-kube-api-access-9zwt2\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.647544 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.647551    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.647776 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.647574    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.647776 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.647653    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.647776 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.647684    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-metrics-client-ca\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.647776 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.647714    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.647776 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.647758    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-tls\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.648047 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.647784    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-grpc-tls\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.648675 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.648647    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-metrics-client-ca\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.650611 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.650582    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.650708 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.650657    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.650708 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.650678    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-grpc-tls\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.650846 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.650777    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-tls\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.650899 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.650867    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.651001 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.650979    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.656278 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.656255    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zwt2\" (UniqueName: \"kubernetes.io/projected/4c501cb9-c2a3-4ba1-a630-5d6e4b959a61-kube-api-access-9zwt2\") pod \"thanos-querier-775c58594c-dv5vh\" (UID: \"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61\") " pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:42.723815 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:42.723736    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:16:43.210144 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:43.210113    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:43.264751 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:43.264714    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:43.536661 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:43.536631    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:43.637128 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:43.637091    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:44.761899 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.761857    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"]
Jul 15 19:16:44.766471 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.764429    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.768567 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.768305    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 15 19:16:44.768567 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.768347    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2bcrh\""
Jul 15 19:16:44.768567 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.768423    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 15 19:16:44.768567 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.768465    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6judud2uo09lm\""
Jul 15 19:16:44.768567 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.768489    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 15 19:16:44.768834 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.768726    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 19:16:44.776606 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.776579    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"]
Jul 15 19:16:44.865980 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.865944    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-audit-log\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.866175 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.866003    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.866175 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.866096    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-secret-metrics-server-client-certs\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.866175 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.866162    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-secret-metrics-server-tls\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.866311 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.866237    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-metrics-server-audit-profiles\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.866311 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.866261    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t897\" (UniqueName: \"kubernetes.io/projected/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-kube-api-access-2t897\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.866384 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.866321    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-client-ca-bundle\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.966846 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.966811    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-audit-log\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.967027 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.966870    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.967027 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.966900    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-secret-metrics-server-client-certs\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.967027 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.966924    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-secret-metrics-server-tls\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.967027 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.966964    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-metrics-server-audit-profiles\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.967027 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.966996    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t897\" (UniqueName: \"kubernetes.io/projected/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-kube-api-access-2t897\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.967314 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.967047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-client-ca-bundle\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.967314 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.967297    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-audit-log\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.967738 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.967689    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.968172 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.968123    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-metrics-server-audit-profiles\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.969776 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.969731    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-secret-metrics-server-client-certs\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.969954 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.969937    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-secret-metrics-server-tls\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.970048 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.970000    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-client-ca-bundle\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:44.985920 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:44.985877    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t897\" (UniqueName: \"kubernetes.io/projected/e61fadd8-c0be-4c32-8b4e-1c660e0549ba-kube-api-access-2t897\") pod \"metrics-server-66c7ff6b65-r8lz2\" (UID: \"e61fadd8-c0be-4c32-8b4e-1c660e0549ba\") " pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:45.078388 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.078306    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:16:45.081653 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.081577    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"]
Jul 15 19:16:45.084079 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.084054    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"
Jul 15 19:16:45.086570 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.086433    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 15 19:16:45.086570 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.086463    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-g88jb\""
Jul 15 19:16:45.093885 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.089806    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cf74b5f5b-8572m"]
Jul 15 19:16:45.093885 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.092550    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"]
Jul 15 19:16:45.112113 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.112083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:45.117012 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.116987    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64d6fd76fc-jktfs"]
Jul 15 19:16:45.119356 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.119338    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.131252 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.131227    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64d6fd76fc-jktfs"]
Jul 15 19:16:45.169789 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.169750    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3935f491-571f-44b7-88d0-8d987b3508bc-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-vmnjq\" (UID: \"3935f491-571f-44b7-88d0-8d987b3508bc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"
Jul 15 19:16:45.169957 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.169818    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nh8s\" (UniqueName: \"kubernetes.io/projected/3935f491-571f-44b7-88d0-8d987b3508bc-kube-api-access-8nh8s\") pod \"monitoring-plugin-5f68df7f58-vmnjq\" (UID: \"3935f491-571f-44b7-88d0-8d987b3508bc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"
Jul 15 19:16:45.219948 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.219918    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:45.270690 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.270656    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nh8s\" (UniqueName: \"kubernetes.io/projected/3935f491-571f-44b7-88d0-8d987b3508bc-kube-api-access-8nh8s\") pod \"monitoring-plugin-5f68df7f58-vmnjq\" (UID: \"3935f491-571f-44b7-88d0-8d987b3508bc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"
Jul 15 19:16:45.270848 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.270708    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-oauth-serving-cert\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.270848 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.270747    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-config\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.270848 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.270791    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-oauth-config\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.270848 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.270819    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-trusted-ca-bundle\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.271054 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.270849    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9fl7\" (UniqueName: \"kubernetes.io/projected/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-kube-api-access-h9fl7\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.271054 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.270873    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-service-ca\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.271054 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.271014    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-serving-cert\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.271169 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.271098    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3935f491-571f-44b7-88d0-8d987b3508bc-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-vmnjq\" (UID: \"3935f491-571f-44b7-88d0-8d987b3508bc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"
Jul 15 19:16:45.271213 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:45.271195    2560 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 15 19:16:45.271278 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:45.271266    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3935f491-571f-44b7-88d0-8d987b3508bc-monitoring-plugin-cert podName:3935f491-571f-44b7-88d0-8d987b3508bc nodeName:}" failed. No retries permitted until 2026-07-15 19:16:45.771246286 +0000 UTC m=+77.841105667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/3935f491-571f-44b7-88d0-8d987b3508bc-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-vmnjq" (UID: "3935f491-571f-44b7-88d0-8d987b3508bc") : secret "monitoring-plugin-cert" not found
Jul 15 19:16:45.280482 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.280443    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nh8s\" (UniqueName: \"kubernetes.io/projected/3935f491-571f-44b7-88d0-8d987b3508bc-kube-api-access-8nh8s\") pod \"monitoring-plugin-5f68df7f58-vmnjq\" (UID: \"3935f491-571f-44b7-88d0-8d987b3508bc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"
Jul 15 19:16:45.371950 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.371853    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-oauth-serving-cert\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.371950 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.371916    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-config\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.372214 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.371961    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-oauth-config\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.372214 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.371990    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-trusted-ca-bundle\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.372214 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.372018    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9fl7\" (UniqueName: \"kubernetes.io/projected/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-kube-api-access-h9fl7\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.372214 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.372043    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-service-ca\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.372214 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.372098    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-serving-cert\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.372903 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.372849    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-config\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.373262 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.373211    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-service-ca\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.373365 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.373294    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-trusted-ca-bundle\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.373539 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.373493    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-oauth-serving-cert\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.374916 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.374892    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-serving-cert\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.375140 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.375124    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-oauth-config\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.381937 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.381913    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9fl7\" (UniqueName: \"kubernetes.io/projected/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-kube-api-access-h9fl7\") pod \"console-64d6fd76fc-jktfs\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") " pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.429214 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.429155    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:16:45.483697 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.483651    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:45.584657 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.584618    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:45.775270 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.775226    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3935f491-571f-44b7-88d0-8d987b3508bc-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-vmnjq\" (UID: \"3935f491-571f-44b7-88d0-8d987b3508bc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"
Jul 15 19:16:45.775735 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:45.775409    2560 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 15 19:16:45.775735 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:16:45.775504    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3935f491-571f-44b7-88d0-8d987b3508bc-monitoring-plugin-cert podName:3935f491-571f-44b7-88d0-8d987b3508bc nodeName:}" failed. No retries permitted until 2026-07-15 19:16:46.775480098 +0000 UTC m=+78.845339485 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/3935f491-571f-44b7-88d0-8d987b3508bc-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-vmnjq" (UID: "3935f491-571f-44b7-88d0-8d987b3508bc") : secret "monitoring-plugin-cert" not found
Jul 15 19:16:45.837790 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.837749    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:45.926736 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.926691    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:45.993389 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:45.993354    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:46.082491 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.082396    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:46.183227 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.183180    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:46.520210 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.519809    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:46.520972 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.520860    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 19:16:46.524541 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.524521    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.527107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.527045    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 19:16:46.527195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.527107    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5rg9r\""
Jul 15 19:16:46.527195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.527107    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 19:16:46.527571 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.527508    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 19:16:46.528437 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.528155    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 19:16:46.528437 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.528170    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 19:16:46.528437 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.528179    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 19:16:46.528437 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.528227    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 19:16:46.528437 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.528172    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3ntpsmmkrac0o\""
Jul 15 19:16:46.528437 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.528315    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 19:16:46.528437 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.528414    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 19:16:46.528858 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.528468    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 19:16:46.528858 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.528718    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 19:16:46.528858 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.528793    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 19:16:46.529231 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.529071    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 19:16:46.530532 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.530371    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 19:16:46.544955 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.544930    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 19:16:46.683589 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.683558    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.683589 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.683590    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxf5j\" (UniqueName: \"kubernetes.io/projected/90459a65-95e1-4d04-b11a-cb616cf8f7c0-kube-api-access-zxf5j\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.683812 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.683622    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.683812 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.683668    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.683812 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.683779    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.683961 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.683822    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.683961 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.683858    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/90459a65-95e1-4d04-b11a-cb616cf8f7c0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.683961 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.683909    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.683961 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.683948    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.684153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.683965    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-config\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.684153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.683990    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.684153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.684020    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.684153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.684039    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-web-config\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.684153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.684063    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.684153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.684079    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.684153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.684101    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.684153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.684130    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.684546 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.684182    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.684546 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.684266    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/90459a65-95e1-4d04-b11a-cb616cf8f7c0-config-out\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.684546 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.684291    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.785697 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.785607    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.785697 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.785655    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-web-config\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.785697 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.785682    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.785697 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.785701    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.785734    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.785765    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.785799    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.785849    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3935f491-571f-44b7-88d0-8d987b3508bc-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-vmnjq\" (UID: \"3935f491-571f-44b7-88d0-8d987b3508bc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.785891    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/90459a65-95e1-4d04-b11a-cb616cf8f7c0-config-out\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.785919    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.785959    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.785985    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxf5j\" (UniqueName: \"kubernetes.io/projected/90459a65-95e1-4d04-b11a-cb616cf8f7c0-kube-api-access-zxf5j\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.786023    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.786047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.786074    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.786105    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.786139    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/90459a65-95e1-4d04-b11a-cb616cf8f7c0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.786171    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786996 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.786206    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786996 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.786237    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-config\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786996 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.786272    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786996 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.786394    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.786996 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.786619    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.788191 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.787614    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.788191 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.787914    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.788344 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.788214    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.789107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.788775    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.789585 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.789558    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.789988 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.789964    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.791871 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.791819    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.792184 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.792119    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/90459a65-95e1-4d04-b11a-cb616cf8f7c0-config-out\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.792532 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.792473    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.792662 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.792639    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.792743 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.792675    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3935f491-571f-44b7-88d0-8d987b3508bc-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-vmnjq\" (UID: \"3935f491-571f-44b7-88d0-8d987b3508bc\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"
Jul 15 19:16:46.793009 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.792986    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/90459a65-95e1-4d04-b11a-cb616cf8f7c0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.793085 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.793017    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.793496 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.793460    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.793579 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.793510    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.793763 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.793738    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-web-config\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.796506 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.795635    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-config\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.797324 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.797282    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.798401 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.798378    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxf5j\" (UniqueName: \"kubernetes.io/projected/90459a65-95e1-4d04-b11a-cb616cf8f7c0-kube-api-access-zxf5j\") pod \"prometheus-k8s-0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.838119 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.838079    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:16:46.903075 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:46.903035    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"
Jul 15 19:16:49.192241 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:49.192208    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:49.261080 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:49.261043    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:50.482876 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:50.482732    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:51.856789 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:51.856747    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:51.909396 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:51.909356    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:54.913002 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:54.912546    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rfhxr" event={"ID":"38246ac2-0ceb-4aef-a07b-50dbc742e79d","Type":"ContainerStarted","Data":"1aa7573d8afcd57343477995b0b072a06e09cac58be8d7c69e4f5c0060c7fd91"}
Jul 15 19:16:55.115332 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.114444    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-m5zgx"]
Jul 15 19:16:55.125177 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.123671    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p"]
Jul 15 19:16:55.164560 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.163714    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-775c58594c-dv5vh"]
Jul 15 19:16:55.185231 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.182797    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8c7fdb7b8-nktpj"]
Jul 15 19:16:55.211849 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.211817    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 19:16:55.275439 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:55.275402    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c88ecb0_35f5_46fb_8b64_31689b50e178.slice/crio-5f023b90e179e82ed8e871fa208f6d03ab4c35fbdb271bf794e9fdfd5e99cd21 WatchSource:0}: Error finding container 5f023b90e179e82ed8e871fa208f6d03ab4c35fbdb271bf794e9fdfd5e99cd21: Status 404 returned error can't find the container with id 5f023b90e179e82ed8e871fa208f6d03ab4c35fbdb271bf794e9fdfd5e99cd21
Jul 15 19:16:55.276158 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:55.276130    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b0da1ee_8857_46b5_a35d_630cb0817dfe.slice/crio-21bab47be094c4b41ed8cdfda6dce88ce93f1af9f7138bbefd869b36a6814790 WatchSource:0}: Error finding container 21bab47be094c4b41ed8cdfda6dce88ce93f1af9f7138bbefd869b36a6814790: Status 404 returned error can't find the container with id 21bab47be094c4b41ed8cdfda6dce88ce93f1af9f7138bbefd869b36a6814790
Jul 15 19:16:55.277792 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:55.277077    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c501cb9_c2a3_4ba1_a630_5d6e4b959a61.slice/crio-0beec56ebe6bd13e39dc34aa81673b52565ea2d02260f743dfe9c5bdcb30e13d WatchSource:0}: Error finding container 0beec56ebe6bd13e39dc34aa81673b52565ea2d02260f743dfe9c5bdcb30e13d: Status 404 returned error can't find the container with id 0beec56ebe6bd13e39dc34aa81673b52565ea2d02260f743dfe9c5bdcb30e13d
Jul 15 19:16:55.280427 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:55.280361    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90459a65_95e1_4d04_b11a_cb616cf8f7c0.slice/crio-22d007cc6eb006cc0734c5c6f34f34a83a12737aaf2cccf6c652cab6f1f26efe WatchSource:0}: Error finding container 22d007cc6eb006cc0734c5c6f34f34a83a12737aaf2cccf6c652cab6f1f26efe: Status 404 returned error can't find the container with id 22d007cc6eb006cc0734c5c6f34f34a83a12737aaf2cccf6c652cab6f1f26efe
Jul 15 19:16:55.305804 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.305775    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 19:16:55.307642 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:55.307623    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76fb1a6_3241_475d_9059_0f347022da7c.slice/crio-c1f3abb774a3b6e8a4bfb6f90ab6a14a6b1055ed2e8921188bbd8cdc776beffa WatchSource:0}: Error finding container c1f3abb774a3b6e8a4bfb6f90ab6a14a6b1055ed2e8921188bbd8cdc776beffa: Status 404 returned error can't find the container with id c1f3abb774a3b6e8a4bfb6f90ab6a14a6b1055ed2e8921188bbd8cdc776beffa
Jul 15 19:16:55.418260 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.418236    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"]
Jul 15 19:16:55.421559 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:55.421285    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3935f491_571f_44b7_88d0_8d987b3508bc.slice/crio-5a8286e8d194b9c2dc8394cddccdbc0ebab9ef7a17001186c962681d8c5b74bc WatchSource:0}: Error finding container 5a8286e8d194b9c2dc8394cddccdbc0ebab9ef7a17001186c962681d8c5b74bc: Status 404 returned error can't find the container with id 5a8286e8d194b9c2dc8394cddccdbc0ebab9ef7a17001186c962681d8c5b74bc
Jul 15 19:16:55.422729 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.422709    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"]
Jul 15 19:16:55.426092 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.426069    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64d6fd76fc-jktfs"]
Jul 15 19:16:55.427899 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:55.427682    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode61fadd8_c0be_4c32_8b4e_1c660e0549ba.slice/crio-1b92987c45dda6eea27e4ccbb8dbc4ea623e214f63af502fca3a822c57d2357e WatchSource:0}: Error finding container 1b92987c45dda6eea27e4ccbb8dbc4ea623e214f63af502fca3a822c57d2357e: Status 404 returned error can't find the container with id 1b92987c45dda6eea27e4ccbb8dbc4ea623e214f63af502fca3a822c57d2357e
Jul 15 19:16:55.429068 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:16:55.428958    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc05e11b_6928_4a9b_a085_90a1d3fcc7dd.slice/crio-19735d8162d1010dfc24bda91299aaddbe3e4f31c4e5620c24059dcb31e2c9ca WatchSource:0}: Error finding container 19735d8162d1010dfc24bda91299aaddbe3e4f31c4e5620c24059dcb31e2c9ca: Status 404 returned error can't find the container with id 19735d8162d1010dfc24bda91299aaddbe3e4f31c4e5620c24059dcb31e2c9ca
Jul 15 19:16:55.923443 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.922503    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c7fdb7b8-nktpj" event={"ID":"1ac8b408-ead4-4902-b9f5-a665c2b35656","Type":"ContainerStarted","Data":"9e99a87a2078a60ddd00bfd84e2d514eaff523c89736d989cbedc83650fdfb4c"}
Jul 15 19:16:55.923443 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.922566    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c7fdb7b8-nktpj" event={"ID":"1ac8b408-ead4-4902-b9f5-a665c2b35656","Type":"ContainerStarted","Data":"df02fb6387dd43badc5d1525f590119125483799d7a06898dffca0c7649d7aab"}
Jul 15 19:16:55.926726 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.926641    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnzkb" event={"ID":"c42c996f-f3ac-450e-bac3-79c20674a579","Type":"ContainerStarted","Data":"9653aca1811381f3b23266ee1c0d12cfe796be8712bd1e9f9ff07f7c04c26287"}
Jul 15 19:16:55.929189 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.929126    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq" event={"ID":"3935f491-571f-44b7-88d0-8d987b3508bc","Type":"ContainerStarted","Data":"5a8286e8d194b9c2dc8394cddccdbc0ebab9ef7a17001186c962681d8c5b74bc"}
Jul 15 19:16:55.931474 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.931375    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerStarted","Data":"c1f3abb774a3b6e8a4bfb6f90ab6a14a6b1055ed2e8921188bbd8cdc776beffa"}
Jul 15 19:16:55.934211 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.934185    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf74b5f5b-8572m" event={"ID":"518fd2e3-df4d-432f-a374-5b28ddfb77f0","Type":"ContainerStarted","Data":"126ca652ed3cc093eff3843fd440439ffde54492a26bd66c4aa422047ff84183"}
Jul 15 19:16:55.936383 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.936262    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx" event={"ID":"1c88ecb0-35f5-46fb-8b64-31689b50e178","Type":"ContainerStarted","Data":"5f023b90e179e82ed8e871fa208f6d03ab4c35fbdb271bf794e9fdfd5e99cd21"}
Jul 15 19:16:55.938797 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.938713    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh" event={"ID":"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61","Type":"ContainerStarted","Data":"0beec56ebe6bd13e39dc34aa81673b52565ea2d02260f743dfe9c5bdcb30e13d"}
Jul 15 19:16:55.940665 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.940642    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2" event={"ID":"e61fadd8-c0be-4c32-8b4e-1c660e0549ba","Type":"ContainerStarted","Data":"1b92987c45dda6eea27e4ccbb8dbc4ea623e214f63af502fca3a822c57d2357e"}
Jul 15 19:16:55.943963 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.942909    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8c7fdb7b8-nktpj" podStartSLOduration=14.942890872 podStartE2EDuration="14.942890872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:55.942210726 +0000 UTC m=+88.012070207" watchObservedRunningTime="2026-07-15 19:16:55.942890872 +0000 UTC m=+88.012750267"
Jul 15 19:16:55.948603 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.947842    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d6fd76fc-jktfs" event={"ID":"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd","Type":"ContainerStarted","Data":"a6f6d6a0bfca8265c21f0afd5d8ebf95946cbe0e14d413cc37e306c21d01ea19"}
Jul 15 19:16:55.948603 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.947904    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d6fd76fc-jktfs" event={"ID":"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd","Type":"ContainerStarted","Data":"19735d8162d1010dfc24bda91299aaddbe3e4f31c4e5620c24059dcb31e2c9ca"}
Jul 15 19:16:55.953400 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.953376    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p" event={"ID":"3b0da1ee-8857-46b5-a35d-630cb0817dfe","Type":"ContainerStarted","Data":"53349003bd9ce6e949ac958fb32d0a4f4affb8a8f56a74449c04703c32d6b9fe"}
Jul 15 19:16:55.953527 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.953432    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p" event={"ID":"3b0da1ee-8857-46b5-a35d-630cb0817dfe","Type":"ContainerStarted","Data":"afccdf4b63fa9a3498ee6bb964bacd836597f4a80c2b1afb65be1d3a13710806"}
Jul 15 19:16:55.953527 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.953473    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p" event={"ID":"3b0da1ee-8857-46b5-a35d-630cb0817dfe","Type":"ContainerStarted","Data":"21bab47be094c4b41ed8cdfda6dce88ce93f1af9f7138bbefd869b36a6814790"}
Jul 15 19:16:55.961871 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.960464    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sk27q" event={"ID":"cb2e4028-856d-4dba-bf82-1d056e6d2da2","Type":"ContainerStarted","Data":"54958f607cd638eda5b02ede50c009e57410e21ff30db2685471423c1dcaeab1"}
Jul 15 19:16:55.961871 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.960521    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sk27q" event={"ID":"cb2e4028-856d-4dba-bf82-1d056e6d2da2","Type":"ContainerStarted","Data":"14c0aca53284500b0af6a2bbea143bef78335cd3ba56d23c007fb8d6e2fbb31b"}
Jul 15 19:16:55.963285 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.963236    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cf74b5f5b-8572m" podStartSLOduration=3.419904538 podStartE2EDuration="19.963216691s" totalImagesPullingTime="16.543312153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:55.958059009 +0000 UTC m=+88.027918407" watchObservedRunningTime="2026-07-15 19:16:55.963216691 +0000 UTC m=+88.033076087"
Jul 15 19:16:55.966710 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.966683    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-djwqm" event={"ID":"995ebd78-54f8-4755-8261-bdca88870130","Type":"ContainerStarted","Data":"50752ee508ee61b2fbfa2f13a0a7a7c27865e726fbec3eb9dd60f4f03549496e"}
Jul 15 19:16:55.972732 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.972631    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerStarted","Data":"22d007cc6eb006cc0734c5c6f34f34a83a12737aaf2cccf6c652cab6f1f26efe"}
Jul 15 19:16:55.978055 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.977745    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-9z7pn" event={"ID":"b8521d92-c8db-4396-9a67-ebb0c3acb319","Type":"ContainerStarted","Data":"7aec86e12dc7b7b867ecda75911d53d1077852df9ceb8304453cfb299fe99ca6"}
Jul 15 19:16:55.978676 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.978658    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-9z7pn"
Jul 15 19:16:55.980297 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.980256    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64d6fd76fc-jktfs" podStartSLOduration=10.980239695 podStartE2EDuration="10.980239695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:55.979174036 +0000 UTC m=+88.049033455" watchObservedRunningTime="2026-07-15 19:16:55.980239695 +0000 UTC m=+88.050099095"
Jul 15 19:16:55.985286 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.985257    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7srhg" event={"ID":"955ee6c7-972b-4215-a5dc-ccdcb08f8da2","Type":"ContainerStarted","Data":"a22df9bf896cb47e136c00e9eb138e321c2670fe340128d94c26e2a8916bdcbf"}
Jul 15 19:16:55.986358 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.986341    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:16:55.991112 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.991096    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-9z7pn"
Jul 15 19:16:55.994019 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:55.993976    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sk27q" podStartSLOduration=70.694674615 podStartE2EDuration="1m26.993958418s" totalImagesPullingTime="16.299283803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:55.993135834 +0000 UTC m=+88.062995232" watchObservedRunningTime="2026-07-15 19:16:55.993958418 +0000 UTC m=+88.063817816"
Jul 15 19:16:56.010585 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:56.010507    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-9z7pn" podStartSLOduration=1.889887979 podStartE2EDuration="27.010487098s" totalImagesPullingTime="25.120599119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:56.008969355 +0000 UTC m=+88.078828752" watchObservedRunningTime="2026-07-15 19:16:56.010487098 +0000 UTC m=+88.080346495"
Jul 15 19:16:56.024445 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:56.024328    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-djwqm" podStartSLOduration=70.475067104 podStartE2EDuration="1m27.024305359s" totalImagesPullingTime="16.549238255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:56.022597651 +0000 UTC m=+88.092457047" watchObservedRunningTime="2026-07-15 19:16:56.024305359 +0000 UTC m=+88.094164755"
Jul 15 19:16:56.111512 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:56.111390    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:56.214044 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:56.212372    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:16:57.238781 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:16:57.238736    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:17:01.317007 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:01.316919    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:01.378638 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:01.378576    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:02.209051 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:02.208964    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:17:02.209051 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:02.209034    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:17:02.215225 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:02.215201    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:17:02.234198 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:02.234148    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7srhg" podStartSLOduration=76.478077255 podStartE2EDuration="1m33.234133009s" totalImagesPullingTime="16.756055754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:56.057884187 +0000 UTC m=+88.127743587" watchObservedRunningTime="2026-07-15 19:17:02.234133009 +0000 UTC m=+94.303992409"
Jul 15 19:17:02.286091 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:02.286014    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:02.389669 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:02.389637    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:03.030351 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:03.030092    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:17:05.038719 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.038687    2560 generic.go:356] "Generic (PLEG): container finished" podID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerID="b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b" exitCode=0
Jul 15 19:17:05.039157 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.038780    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerDied","Data":"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"}
Jul 15 19:17:05.040655 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.040627    2560 generic.go:356] "Generic (PLEG): container finished" podID="38246ac2-0ceb-4aef-a07b-50dbc742e79d" containerID="11dcc31cece2252309d3796d6465f93a76f5db75f3432c01e666f3d1f699e369" exitCode=0
Jul 15 19:17:05.040787 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.040721    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rfhxr" event={"ID":"38246ac2-0ceb-4aef-a07b-50dbc742e79d","Type":"ContainerDied","Data":"11dcc31cece2252309d3796d6465f93a76f5db75f3432c01e666f3d1f699e369"}
Jul 15 19:17:05.042858 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.042836    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnzkb" event={"ID":"c42c996f-f3ac-450e-bac3-79c20674a579","Type":"ContainerStarted","Data":"44dc9fe7c98398171e611b33f5e5e328160d81310016f540802d47d323b62ef2"}
Jul 15 19:17:05.044227 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.044210    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq" event={"ID":"3935f491-571f-44b7-88d0-8d987b3508bc","Type":"ContainerStarted","Data":"0c8897c710d821f0959e95a08fc624dcd2c0b30674d3c2dfb98741ad47d54777"}
Jul 15 19:17:05.044401 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.044255    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"
Jul 15 19:17:05.045813 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.045791    2560 generic.go:356] "Generic (PLEG): container finished" podID="d76fb1a6-3241-475d-9059-0f347022da7c" containerID="29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4" exitCode=0
Jul 15 19:17:05.045900 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.045874    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerDied","Data":"29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4"}
Jul 15 19:17:05.050135 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.050109    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx" event={"ID":"1c88ecb0-35f5-46fb-8b64-31689b50e178","Type":"ContainerStarted","Data":"c08e0eca837cf74bcac881104d7bb82b771f52d4ebc039b6868a729a4c18190b"}
Jul 15 19:17:05.050227 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.050167    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx" event={"ID":"1c88ecb0-35f5-46fb-8b64-31689b50e178","Type":"ContainerStarted","Data":"60f107a93b7d45b7ae20684efad180ae14e7dc6f0c46286403e4d5f15a402fa2"}
Jul 15 19:17:05.050227 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.050199    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx" event={"ID":"1c88ecb0-35f5-46fb-8b64-31689b50e178","Type":"ContainerStarted","Data":"4b32960885d0b1e501fcb7bdf282a1dec5d0fb261ec97f29221ec5fae1b9d8ea"}
Jul 15 19:17:05.051498 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.051445    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq"
Jul 15 19:17:05.053636 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.053611    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh" event={"ID":"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61","Type":"ContainerStarted","Data":"ac5bdbabba05be0e0693b6b48283508eb656b33bb6ae295ebf015124adf50ac3"}
Jul 15 19:17:05.053732 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.053661    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh" event={"ID":"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61","Type":"ContainerStarted","Data":"cfa3ace94f796c1731434a67e06f49cba1508787a93c1f64bb9ca07a874563d8"}
Jul 15 19:17:05.053732 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.053692    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh" event={"ID":"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61","Type":"ContainerStarted","Data":"f289eb79eadaae616e8b512564e32a0d11aa02d4e72e078bc4bd050ce198bef7"}
Jul 15 19:17:05.055102 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.055082    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2" event={"ID":"e61fadd8-c0be-4c32-8b4e-1c660e0549ba","Type":"ContainerStarted","Data":"70438db4248a2ddd35569b9cca636e7d5e8b1bcbad085813e3cbab8de83741b3"}
Jul 15 19:17:05.057153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.057135    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p" event={"ID":"3b0da1ee-8857-46b5-a35d-630cb0817dfe","Type":"ContainerStarted","Data":"11b7270092888a478e1ee2560326c15091b9939b6514de490917492c1b2c6f60"}
Jul 15 19:17:05.078729 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.078604    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:17:05.078729 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.078642    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:17:05.086580 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.086543    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-m5zgx" podStartSLOduration=15.980327595 podStartE2EDuration="25.086530031s" totalImagesPullingTime="9.106202436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:17:05.085356953 +0000 UTC m=+97.155216350" watchObservedRunningTime="2026-07-15 19:17:05.086530031 +0000 UTC m=+97.156389427"
Jul 15 19:17:05.145540 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.145495    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hnzkb" podStartSLOduration=3.730053805 podStartE2EDuration="36.145477958s" totalImagesPullingTime="32.415424153s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:16:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:17:05.144101107 +0000 UTC m=+97.213960498" watchObservedRunningTime="2026-07-15 19:17:05.145477958 +0000 UTC m=+97.215337355"
Jul 15 19:17:05.163777 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.163734    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2" podStartSLOduration=12.256651571 podStartE2EDuration="21.163718131s" totalImagesPullingTime="8.90706656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:17:05.162843188 +0000 UTC m=+97.232702575" watchObservedRunningTime="2026-07-15 19:17:05.163718131 +0000 UTC m=+97.233577506"
Jul 15 19:17:05.177888 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.177801    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-vmnjq" podStartSLOduration=11.219149976 podStartE2EDuration="20.177782402s" totalImagesPullingTime="8.958632426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:17:05.177022611 +0000 UTC m=+97.246882010" watchObservedRunningTime="2026-07-15 19:17:05.177782402 +0000 UTC m=+97.247641800"
Jul 15 19:17:05.193733 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.193678    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdm2p" podStartSLOduration=16.407841177 podStartE2EDuration="25.193658493s" totalImagesPullingTime="8.785817316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:17:05.192677438 +0000 UTC m=+97.262536841" watchObservedRunningTime="2026-07-15 19:17:05.193658493 +0000 UTC m=+97.263517890"
Jul 15 19:17:05.429668 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.429571    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:17:05.429668 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.429635    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:17:05.435038 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.435009    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:17:05.935010 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.934974    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:05.991035 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:05.990991    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:06.065998 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.065959    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rfhxr" event={"ID":"38246ac2-0ceb-4aef-a07b-50dbc742e79d","Type":"ContainerStarted","Data":"0a88b8a41a8ad4eb3b049d8baaf8fb8e530819ea488638c8d225e2672dd5f15c"}
Jul 15 19:17:06.066510 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.066018    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rfhxr" event={"ID":"38246ac2-0ceb-4aef-a07b-50dbc742e79d","Type":"ContainerStarted","Data":"077b88154d9ee0723151df6d63bc11b587eb58cb19e6c48f85a8de41b5014fa9"}
Jul 15 19:17:06.072426 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.072371    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:17:06.084277 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.084242    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-96b457b94-jz987" podUID="9356193f-ee01-4c43-8ea1-7550f4b082af" containerName="registry" containerID="cri-o://bc3061d5729ebf86840a71a59e999c6afd965ae69d9bff1b8ca594f74cdd4d9e" gracePeriod=30
Jul 15 19:17:06.087402 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.087277    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rfhxr" podStartSLOduration=19.720040351 podStartE2EDuration="26.087257731s" totalImagesPullingTime="6.26723644s" totalInitContainerRuntime="99.98094ms" isStatefulPod=true podCreationTimestamp="2026-07-15 19:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:17:06.086813935 +0000 UTC m=+98.156673334" watchObservedRunningTime="2026-07-15 19:17:06.087257731 +0000 UTC m=+98.157117128"
Jul 15 19:17:06.135163 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.135138    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8c7fdb7b8-nktpj"]
Jul 15 19:17:06.408003 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.407973    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:17:06.495525 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.495495    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-bound-sa-token\") pod \"9356193f-ee01-4c43-8ea1-7550f4b082af\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") "
Jul 15 19:17:06.495706 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.495533    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9356193f-ee01-4c43-8ea1-7550f4b082af-image-registry-private-configuration\") pod \"9356193f-ee01-4c43-8ea1-7550f4b082af\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") "
Jul 15 19:17:06.495706 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.495560    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9356193f-ee01-4c43-8ea1-7550f4b082af-installation-pull-secrets\") pod \"9356193f-ee01-4c43-8ea1-7550f4b082af\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") "
Jul 15 19:17:06.495706 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.495592    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-registry-tls\") pod \"9356193f-ee01-4c43-8ea1-7550f4b082af\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") "
Jul 15 19:17:06.495706 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.495647    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9356193f-ee01-4c43-8ea1-7550f4b082af-ca-trust-extracted\") pod \"9356193f-ee01-4c43-8ea1-7550f4b082af\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") "
Jul 15 19:17:06.495706 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.495674    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmvdr\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-kube-api-access-kmvdr\") pod \"9356193f-ee01-4c43-8ea1-7550f4b082af\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") "
Jul 15 19:17:06.495706 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.495696    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9356193f-ee01-4c43-8ea1-7550f4b082af-trusted-ca\") pod \"9356193f-ee01-4c43-8ea1-7550f4b082af\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") "
Jul 15 19:17:06.495998 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.495742    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9356193f-ee01-4c43-8ea1-7550f4b082af-registry-certificates\") pod \"9356193f-ee01-4c43-8ea1-7550f4b082af\" (UID: \"9356193f-ee01-4c43-8ea1-7550f4b082af\") "
Jul 15 19:17:06.496609 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.496335    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9356193f-ee01-4c43-8ea1-7550f4b082af-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9356193f-ee01-4c43-8ea1-7550f4b082af" (UID: "9356193f-ee01-4c43-8ea1-7550f4b082af"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:17:06.496609 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.496490    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9356193f-ee01-4c43-8ea1-7550f4b082af-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9356193f-ee01-4c43-8ea1-7550f4b082af" (UID: "9356193f-ee01-4c43-8ea1-7550f4b082af"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:17:06.498823 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.498785    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9356193f-ee01-4c43-8ea1-7550f4b082af-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9356193f-ee01-4c43-8ea1-7550f4b082af" (UID: "9356193f-ee01-4c43-8ea1-7550f4b082af"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:17:06.498993 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.498961    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9356193f-ee01-4c43-8ea1-7550f4b082af-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9356193f-ee01-4c43-8ea1-7550f4b082af" (UID: "9356193f-ee01-4c43-8ea1-7550f4b082af"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:17:06.499091 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.499031    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9356193f-ee01-4c43-8ea1-7550f4b082af" (UID: "9356193f-ee01-4c43-8ea1-7550f4b082af"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:17:06.499269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.499227    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-kube-api-access-kmvdr" (OuterVolumeSpecName: "kube-api-access-kmvdr") pod "9356193f-ee01-4c43-8ea1-7550f4b082af" (UID: "9356193f-ee01-4c43-8ea1-7550f4b082af"). InnerVolumeSpecName "kube-api-access-kmvdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:17:06.499342 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.499309    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9356193f-ee01-4c43-8ea1-7550f4b082af" (UID: "9356193f-ee01-4c43-8ea1-7550f4b082af"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:17:06.506144 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.506115    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:06.507503 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.507477    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9356193f-ee01-4c43-8ea1-7550f4b082af-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9356193f-ee01-4c43-8ea1-7550f4b082af" (UID: "9356193f-ee01-4c43-8ea1-7550f4b082af"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:17:06.563660 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.563623    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:06.596975 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.596945    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmvdr\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-kube-api-access-kmvdr\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:06.596975 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.596976    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9356193f-ee01-4c43-8ea1-7550f4b082af-trusted-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:06.597180 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.596993    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9356193f-ee01-4c43-8ea1-7550f4b082af-registry-certificates\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:06.597180 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.597006    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-bound-sa-token\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:06.597180 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.597021    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9356193f-ee01-4c43-8ea1-7550f4b082af-image-registry-private-configuration\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:06.597180 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.597034    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9356193f-ee01-4c43-8ea1-7550f4b082af-installation-pull-secrets\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:06.597180 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.597048    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9356193f-ee01-4c43-8ea1-7550f4b082af-registry-tls\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:06.597180 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.597060    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9356193f-ee01-4c43-8ea1-7550f4b082af-ca-trust-extracted\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:06.644726 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:06.644695    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:07.071624 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:07.071586    2560 generic.go:356] "Generic (PLEG): container finished" podID="9356193f-ee01-4c43-8ea1-7550f4b082af" containerID="bc3061d5729ebf86840a71a59e999c6afd965ae69d9bff1b8ca594f74cdd4d9e" exitCode=0
Jul 15 19:17:07.072096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:07.071637    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96b457b94-jz987"
Jul 15 19:17:07.072096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:07.071668    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-96b457b94-jz987" event={"ID":"9356193f-ee01-4c43-8ea1-7550f4b082af","Type":"ContainerDied","Data":"bc3061d5729ebf86840a71a59e999c6afd965ae69d9bff1b8ca594f74cdd4d9e"}
Jul 15 19:17:07.072096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:07.071726    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-96b457b94-jz987" event={"ID":"9356193f-ee01-4c43-8ea1-7550f4b082af","Type":"ContainerDied","Data":"5424fec57b20442000cbfc321a635730567c519b2508218d36cdd021f99e8bd8"}
Jul 15 19:17:07.072096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:07.071760    2560 scope.go:117] "RemoveContainer" containerID="bc3061d5729ebf86840a71a59e999c6afd965ae69d9bff1b8ca594f74cdd4d9e"
Jul 15 19:17:07.081033 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:07.081015    2560 scope.go:117] "RemoveContainer" containerID="bc3061d5729ebf86840a71a59e999c6afd965ae69d9bff1b8ca594f74cdd4d9e"
Jul 15 19:17:07.081280 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:17:07.081259    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc3061d5729ebf86840a71a59e999c6afd965ae69d9bff1b8ca594f74cdd4d9e\": container with ID starting with bc3061d5729ebf86840a71a59e999c6afd965ae69d9bff1b8ca594f74cdd4d9e not found: ID does not exist" containerID="bc3061d5729ebf86840a71a59e999c6afd965ae69d9bff1b8ca594f74cdd4d9e"
Jul 15 19:17:07.081345 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:07.081292    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc3061d5729ebf86840a71a59e999c6afd965ae69d9bff1b8ca594f74cdd4d9e"} err="failed to get container status \"bc3061d5729ebf86840a71a59e999c6afd965ae69d9bff1b8ca594f74cdd4d9e\": rpc error: code = NotFound desc = could not find container \"bc3061d5729ebf86840a71a59e999c6afd965ae69d9bff1b8ca594f74cdd4d9e\": container with ID starting with bc3061d5729ebf86840a71a59e999c6afd965ae69d9bff1b8ca594f74cdd4d9e not found: ID does not exist"
Jul 15 19:17:07.092622 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:07.092600    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-96b457b94-jz987"]
Jul 15 19:17:07.095869 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:07.095849    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-96b457b94-jz987"]
Jul 15 19:17:08.495809 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:08.495767    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9356193f-ee01-4c43-8ea1-7550f4b082af" path="/var/lib/kubelet/pods/9356193f-ee01-4c43-8ea1-7550f4b082af/volumes"
Jul 15 19:17:10.091596 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:10.091530    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerStarted","Data":"a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f"}
Jul 15 19:17:10.091946 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:10.091645    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerStarted","Data":"b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd"}
Jul 15 19:17:10.091946 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:10.091699    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerStarted","Data":"b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83"}
Jul 15 19:17:10.094747 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:10.094725    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh" event={"ID":"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61","Type":"ContainerStarted","Data":"02dcbbc203786aa11354edb01e31d7d35e355728f7c11cdfa09ed38528010314"}
Jul 15 19:17:10.094855 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:10.094810    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh" event={"ID":"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61","Type":"ContainerStarted","Data":"476ac56ff99c884fbe073690bf91465e892eac5b5d47214b5385ab52eb69dd07"}
Jul 15 19:17:10.097469 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:10.097429    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerStarted","Data":"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"}
Jul 15 19:17:10.097561 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:10.097511    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerStarted","Data":"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"}
Jul 15 19:17:10.097561 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:10.097545    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerStarted","Data":"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"}
Jul 15 19:17:10.097637 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:10.097574    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerStarted","Data":"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"}
Jul 15 19:17:11.105550 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:11.105512    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerStarted","Data":"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"}
Jul 15 19:17:11.106159 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:11.105592    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerStarted","Data":"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"}
Jul 15 19:17:11.108567 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:11.108538    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerStarted","Data":"83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515"}
Jul 15 19:17:11.108692 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:11.108612    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerStarted","Data":"e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410"}
Jul 15 19:17:11.108692 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:11.108649    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerStarted","Data":"c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b"}
Jul 15 19:17:11.111489 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:11.111464    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh" event={"ID":"4c501cb9-c2a3-4ba1-a630-5d6e4b959a61","Type":"ContainerStarted","Data":"b8bcc17dc04ab0deea8072066783d18d2b8c5aedb29fcfa3db84196c4e60d8e7"}
Jul 15 19:17:11.111710 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:11.111693    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:17:11.116629 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:11.116608    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh"
Jul 15 19:17:11.134679 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:11.134632    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=11.754093291 podStartE2EDuration="25.134615076s" totalImagesPullingTime="13.344177397s" totalInitContainerRuntime="36.344388ms" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:17:11.132099538 +0000 UTC m=+103.201958946" watchObservedRunningTime="2026-07-15 19:17:11.134615076 +0000 UTC m=+103.204474474"
Jul 15 19:17:11.167692 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:11.167643    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=16.727786255 podStartE2EDuration="30.167626812s" totalImagesPullingTime="13.407954646s" totalInitContainerRuntime="31.885911ms" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:17:11.164548706 +0000 UTC m=+103.234408102" watchObservedRunningTime="2026-07-15 19:17:11.167626812 +0000 UTC m=+103.237486209"
Jul 15 19:17:11.186676 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:11.186618    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-775c58594c-dv5vh" podStartSLOduration=15.241795842 podStartE2EDuration="29.186600308s" totalImagesPullingTime="13.944804466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:17:11.185167831 +0000 UTC m=+103.255027239" watchObservedRunningTime="2026-07-15 19:17:11.186600308 +0000 UTC m=+103.256459706"
Jul 15 19:17:11.838321 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:11.838282    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:17:15.061856 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:15.061824    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:15.114654 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:15.114612    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:16.320370 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:16.320332    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:16.418309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:16.418277    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:20.966569 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:20.966524    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5cf74b5f5b-8572m" podUID="518fd2e3-df4d-432f-a374-5b28ddfb77f0" containerName="console" containerID="cri-o://126ca652ed3cc093eff3843fd440439ffde54492a26bd66c4aa422047ff84183" gracePeriod=15
Jul 15 19:17:21.161286 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.161248    2560 generic.go:356] "Generic (PLEG): container finished" podID="518fd2e3-df4d-432f-a374-5b28ddfb77f0" containerID="126ca652ed3cc093eff3843fd440439ffde54492a26bd66c4aa422047ff84183" exitCode=0
Jul 15 19:17:21.161412 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.161318    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf74b5f5b-8572m" event={"ID":"518fd2e3-df4d-432f-a374-5b28ddfb77f0","Type":"ContainerDied","Data":"126ca652ed3cc093eff3843fd440439ffde54492a26bd66c4aa422047ff84183"}
Jul 15 19:17:21.257550 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.257524    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:17:21.328189 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.328158    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-serving-cert\") pod \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") "
Jul 15 19:17:21.328339 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.328202    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-service-ca\") pod \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") "
Jul 15 19:17:21.328339 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.328224    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-oauth-serving-cert\") pod \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") "
Jul 15 19:17:21.328339 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.328263    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78vwf\" (UniqueName: \"kubernetes.io/projected/518fd2e3-df4d-432f-a374-5b28ddfb77f0-kube-api-access-78vwf\") pod \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") "
Jul 15 19:17:21.328339 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.328292    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-config\") pod \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") "
Jul 15 19:17:21.328339 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.328337    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-oauth-config\") pod \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\" (UID: \"518fd2e3-df4d-432f-a374-5b28ddfb77f0\") "
Jul 15 19:17:21.328849 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.328795    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "518fd2e3-df4d-432f-a374-5b28ddfb77f0" (UID: "518fd2e3-df4d-432f-a374-5b28ddfb77f0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:17:21.328849 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.328805    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-config" (OuterVolumeSpecName: "console-config") pod "518fd2e3-df4d-432f-a374-5b28ddfb77f0" (UID: "518fd2e3-df4d-432f-a374-5b28ddfb77f0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:17:21.328994 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.328905    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-service-ca" (OuterVolumeSpecName: "service-ca") pod "518fd2e3-df4d-432f-a374-5b28ddfb77f0" (UID: "518fd2e3-df4d-432f-a374-5b28ddfb77f0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:17:21.330748 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.330720    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "518fd2e3-df4d-432f-a374-5b28ddfb77f0" (UID: "518fd2e3-df4d-432f-a374-5b28ddfb77f0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:17:21.330748 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.330738    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/518fd2e3-df4d-432f-a374-5b28ddfb77f0-kube-api-access-78vwf" (OuterVolumeSpecName: "kube-api-access-78vwf") pod "518fd2e3-df4d-432f-a374-5b28ddfb77f0" (UID: "518fd2e3-df4d-432f-a374-5b28ddfb77f0"). InnerVolumeSpecName "kube-api-access-78vwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:17:21.330893 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.330765    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "518fd2e3-df4d-432f-a374-5b28ddfb77f0" (UID: "518fd2e3-df4d-432f-a374-5b28ddfb77f0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:17:21.429419 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.429387    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-serving-cert\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:21.429419 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.429418    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-service-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:21.429607 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.429432    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-oauth-serving-cert\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:21.429607 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.429444    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78vwf\" (UniqueName: \"kubernetes.io/projected/518fd2e3-df4d-432f-a374-5b28ddfb77f0-kube-api-access-78vwf\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:21.429607 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.429477    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:21.429607 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:21.429489    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/518fd2e3-df4d-432f-a374-5b28ddfb77f0-console-oauth-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:22.166435 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:22.166408    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf74b5f5b-8572m"
Jul 15 19:17:22.166849 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:22.166411    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf74b5f5b-8572m" event={"ID":"518fd2e3-df4d-432f-a374-5b28ddfb77f0","Type":"ContainerDied","Data":"691a082080c2ec8fcf359a211601d93a8282a32c033dbbc26ea59a93cdd65b8c"}
Jul 15 19:17:22.166849 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:22.166530    2560 scope.go:117] "RemoveContainer" containerID="126ca652ed3cc093eff3843fd440439ffde54492a26bd66c4aa422047ff84183"
Jul 15 19:17:22.189017 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:22.188991    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cf74b5f5b-8572m"]
Jul 15 19:17:22.193393 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:22.193373    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cf74b5f5b-8572m"]
Jul 15 19:17:22.494857 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:22.494821    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="518fd2e3-df4d-432f-a374-5b28ddfb77f0" path="/var/lib/kubelet/pods/518fd2e3-df4d-432f-a374-5b28ddfb77f0/volumes"
Jul 15 19:17:23.897062 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:23.897035    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:23.997868 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:23.997834    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:24.052965 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:24.052933    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:24.179137 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:24.179061    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:24.236536 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:24.236507    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:25.083956 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:25.083925    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:17:25.087725 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:25.087696    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-66c7ff6b65-r8lz2"
Jul 15 19:17:26.993818 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:26.993788    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7srhg"
Jul 15 19:17:31.154898 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.154821    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8c7fdb7b8-nktpj" podUID="1ac8b408-ead4-4902-b9f5-a665c2b35656" containerName="console" containerID="cri-o://9e99a87a2078a60ddd00bfd84e2d514eaff523c89736d989cbedc83650fdfb4c" gracePeriod=15
Jul 15 19:17:31.318320 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.318292    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:31.374899 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.374854    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:31.407799 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.407740    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:17:31.445723 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.445695    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbh5x\" (UniqueName: \"kubernetes.io/projected/1ac8b408-ead4-4902-b9f5-a665c2b35656-kube-api-access-zbh5x\") pod \"1ac8b408-ead4-4902-b9f5-a665c2b35656\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") "
Jul 15 19:17:31.445847 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.445731    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-service-ca\") pod \"1ac8b408-ead4-4902-b9f5-a665c2b35656\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") "
Jul 15 19:17:31.445847 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.445773    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-oauth-serving-cert\") pod \"1ac8b408-ead4-4902-b9f5-a665c2b35656\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") "
Jul 15 19:17:31.445847 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.445805    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-oauth-config\") pod \"1ac8b408-ead4-4902-b9f5-a665c2b35656\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") "
Jul 15 19:17:31.445847 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.445831    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-trusted-ca-bundle\") pod \"1ac8b408-ead4-4902-b9f5-a665c2b35656\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") "
Jul 15 19:17:31.446040 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.445906    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-config\") pod \"1ac8b408-ead4-4902-b9f5-a665c2b35656\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") "
Jul 15 19:17:31.446040 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.445955    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-serving-cert\") pod \"1ac8b408-ead4-4902-b9f5-a665c2b35656\" (UID: \"1ac8b408-ead4-4902-b9f5-a665c2b35656\") "
Jul 15 19:17:31.446248 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.446204    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-service-ca" (OuterVolumeSpecName: "service-ca") pod "1ac8b408-ead4-4902-b9f5-a665c2b35656" (UID: "1ac8b408-ead4-4902-b9f5-a665c2b35656"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:17:31.446419 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.446357    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1ac8b408-ead4-4902-b9f5-a665c2b35656" (UID: "1ac8b408-ead4-4902-b9f5-a665c2b35656"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:17:31.446419 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.446364    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1ac8b408-ead4-4902-b9f5-a665c2b35656" (UID: "1ac8b408-ead4-4902-b9f5-a665c2b35656"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:17:31.446419 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.446414    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-config" (OuterVolumeSpecName: "console-config") pod "1ac8b408-ead4-4902-b9f5-a665c2b35656" (UID: "1ac8b408-ead4-4902-b9f5-a665c2b35656"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:17:31.448034 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.448002    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1ac8b408-ead4-4902-b9f5-a665c2b35656" (UID: "1ac8b408-ead4-4902-b9f5-a665c2b35656"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:17:31.448131 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.448113    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1ac8b408-ead4-4902-b9f5-a665c2b35656" (UID: "1ac8b408-ead4-4902-b9f5-a665c2b35656"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:17:31.448322 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.448305    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ac8b408-ead4-4902-b9f5-a665c2b35656-kube-api-access-zbh5x" (OuterVolumeSpecName: "kube-api-access-zbh5x") pod "1ac8b408-ead4-4902-b9f5-a665c2b35656" (UID: "1ac8b408-ead4-4902-b9f5-a665c2b35656"). InnerVolumeSpecName "kube-api-access-zbh5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:17:31.547028 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.546996    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-oauth-serving-cert\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:31.547028 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.547024    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-oauth-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:31.547028 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.547033    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-trusted-ca-bundle\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:31.547231 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.547042    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:31.547231 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.547052    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ac8b408-ead4-4902-b9f5-a665c2b35656-console-serving-cert\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:31.547231 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.547060    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbh5x\" (UniqueName: \"kubernetes.io/projected/1ac8b408-ead4-4902-b9f5-a665c2b35656-kube-api-access-zbh5x\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:31.547231 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:31.547069    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ac8b408-ead4-4902-b9f5-a665c2b35656-service-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:17:32.207233 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:32.207196    2560 generic.go:356] "Generic (PLEG): container finished" podID="1ac8b408-ead4-4902-b9f5-a665c2b35656" containerID="9e99a87a2078a60ddd00bfd84e2d514eaff523c89736d989cbedc83650fdfb4c" exitCode=0
Jul 15 19:17:32.207715 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:32.207239    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c7fdb7b8-nktpj"
Jul 15 19:17:32.207715 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:32.207255    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c7fdb7b8-nktpj" event={"ID":"1ac8b408-ead4-4902-b9f5-a665c2b35656","Type":"ContainerDied","Data":"9e99a87a2078a60ddd00bfd84e2d514eaff523c89736d989cbedc83650fdfb4c"}
Jul 15 19:17:32.207715 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:32.207299    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c7fdb7b8-nktpj" event={"ID":"1ac8b408-ead4-4902-b9f5-a665c2b35656","Type":"ContainerDied","Data":"df02fb6387dd43badc5d1525f590119125483799d7a06898dffca0c7649d7aab"}
Jul 15 19:17:32.207715 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:32.207320    2560 scope.go:117] "RemoveContainer" containerID="9e99a87a2078a60ddd00bfd84e2d514eaff523c89736d989cbedc83650fdfb4c"
Jul 15 19:17:32.215604 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:32.215581    2560 scope.go:117] "RemoveContainer" containerID="9e99a87a2078a60ddd00bfd84e2d514eaff523c89736d989cbedc83650fdfb4c"
Jul 15 19:17:32.215863 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:17:32.215844    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e99a87a2078a60ddd00bfd84e2d514eaff523c89736d989cbedc83650fdfb4c\": container with ID starting with 9e99a87a2078a60ddd00bfd84e2d514eaff523c89736d989cbedc83650fdfb4c not found: ID does not exist" containerID="9e99a87a2078a60ddd00bfd84e2d514eaff523c89736d989cbedc83650fdfb4c"
Jul 15 19:17:32.215926 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:32.215874    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e99a87a2078a60ddd00bfd84e2d514eaff523c89736d989cbedc83650fdfb4c"} err="failed to get container status \"9e99a87a2078a60ddd00bfd84e2d514eaff523c89736d989cbedc83650fdfb4c\": rpc error: code = NotFound desc = could not find container \"9e99a87a2078a60ddd00bfd84e2d514eaff523c89736d989cbedc83650fdfb4c\": container with ID starting with 9e99a87a2078a60ddd00bfd84e2d514eaff523c89736d989cbedc83650fdfb4c not found: ID does not exist"
Jul 15 19:17:32.227327 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:32.227304    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8c7fdb7b8-nktpj"]
Jul 15 19:17:32.231028 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:32.231005    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8c7fdb7b8-nktpj"]
Jul 15 19:17:32.493986 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:32.493954    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ac8b408-ead4-4902-b9f5-a665c2b35656" path="/var/lib/kubelet/pods/1ac8b408-ead4-4902-b9f5-a665c2b35656/volumes"
Jul 15 19:17:46.838464 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:46.838419    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:17:46.901660 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:46.901636    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:17:47.283222 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:47.283195    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:17:59.470263 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:59.470225    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:59.741631 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:59.741597    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:17:59.962076 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:17:59.962043    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:18:00.018367 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:00.018276    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:18:00.540135 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:00.540099    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 19:18:00.540576 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:00.540498    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="alertmanager" containerID="cri-o://b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83" gracePeriod=120
Jul 15 19:18:00.540576 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:00.540521    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="kube-rbac-proxy-web" containerID="cri-o://a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f" gracePeriod=120
Jul 15 19:18:00.540576 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:00.540518    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="kube-rbac-proxy-metric" containerID="cri-o://c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b" gracePeriod=120
Jul 15 19:18:00.540734 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:00.540562    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="config-reloader" containerID="cri-o://b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd" gracePeriod=120
Jul 15 19:18:00.540734 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:00.540588    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="kube-rbac-proxy" containerID="cri-o://83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515" gracePeriod=120
Jul 15 19:18:00.540734 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:00.540586    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="prom-label-proxy" containerID="cri-o://e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410" gracePeriod=120
Jul 15 19:18:01.327265 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.327234    2560 generic.go:356] "Generic (PLEG): container finished" podID="d76fb1a6-3241-475d-9059-0f347022da7c" containerID="e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410" exitCode=0
Jul 15 19:18:01.327265 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.327258    2560 generic.go:356] "Generic (PLEG): container finished" podID="d76fb1a6-3241-475d-9059-0f347022da7c" containerID="83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515" exitCode=0
Jul 15 19:18:01.327265 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.327264    2560 generic.go:356] "Generic (PLEG): container finished" podID="d76fb1a6-3241-475d-9059-0f347022da7c" containerID="b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd" exitCode=0
Jul 15 19:18:01.327265 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.327269    2560 generic.go:356] "Generic (PLEG): container finished" podID="d76fb1a6-3241-475d-9059-0f347022da7c" containerID="b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83" exitCode=0
Jul 15 19:18:01.327543 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.327301    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerDied","Data":"e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410"}
Jul 15 19:18:01.327543 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.327364    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerDied","Data":"83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515"}
Jul 15 19:18:01.327543 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.327390    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerDied","Data":"b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd"}
Jul 15 19:18:01.327543 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.327415    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerDied","Data":"b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83"}
Jul 15 19:18:01.804748 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.804721    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:18:01.897234 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.897126    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy-web\") pod \"d76fb1a6-3241-475d-9059-0f347022da7c\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") "
Jul 15 19:18:01.897234 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.897198    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d76fb1a6-3241-475d-9059-0f347022da7c\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") "
Jul 15 19:18:01.897234 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.897229    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-cluster-tls-config\") pod \"d76fb1a6-3241-475d-9059-0f347022da7c\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") "
Jul 15 19:18:01.897537 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.897372    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-config-volume\") pod \"d76fb1a6-3241-475d-9059-0f347022da7c\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") "
Jul 15 19:18:01.897537 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.897410    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d76fb1a6-3241-475d-9059-0f347022da7c-tls-assets\") pod \"d76fb1a6-3241-475d-9059-0f347022da7c\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") "
Jul 15 19:18:01.897537 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.897482    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-web-config\") pod \"d76fb1a6-3241-475d-9059-0f347022da7c\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") "
Jul 15 19:18:01.897537 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.897522    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy\") pod \"d76fb1a6-3241-475d-9059-0f347022da7c\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") "
Jul 15 19:18:01.897774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.897548    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d76fb1a6-3241-475d-9059-0f347022da7c-config-out\") pod \"d76fb1a6-3241-475d-9059-0f347022da7c\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") "
Jul 15 19:18:01.897774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.897586    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d76fb1a6-3241-475d-9059-0f347022da7c-metrics-client-ca\") pod \"d76fb1a6-3241-475d-9059-0f347022da7c\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") "
Jul 15 19:18:01.897774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.897628    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9j4z4\" (UniqueName: \"kubernetes.io/projected/d76fb1a6-3241-475d-9059-0f347022da7c-kube-api-access-9j4z4\") pod \"d76fb1a6-3241-475d-9059-0f347022da7c\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") "
Jul 15 19:18:01.897774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.897681    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d76fb1a6-3241-475d-9059-0f347022da7c-alertmanager-trusted-ca-bundle\") pod \"d76fb1a6-3241-475d-9059-0f347022da7c\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") "
Jul 15 19:18:01.897774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.897712    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-main-tls\") pod \"d76fb1a6-3241-475d-9059-0f347022da7c\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") "
Jul 15 19:18:01.897774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.897752    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d76fb1a6-3241-475d-9059-0f347022da7c-alertmanager-main-db\") pod \"d76fb1a6-3241-475d-9059-0f347022da7c\" (UID: \"d76fb1a6-3241-475d-9059-0f347022da7c\") "
Jul 15 19:18:01.898384 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.898312    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d76fb1a6-3241-475d-9059-0f347022da7c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d76fb1a6-3241-475d-9059-0f347022da7c" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:18:01.898763 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.898715    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d76fb1a6-3241-475d-9059-0f347022da7c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d76fb1a6-3241-475d-9059-0f347022da7c" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:18:01.898866 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.898830    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d76fb1a6-3241-475d-9059-0f347022da7c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d76fb1a6-3241-475d-9059-0f347022da7c" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:18:01.900439 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.900410    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-config-volume" (OuterVolumeSpecName: "config-volume") pod "d76fb1a6-3241-475d-9059-0f347022da7c" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:01.900732 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.900684    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d76fb1a6-3241-475d-9059-0f347022da7c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d76fb1a6-3241-475d-9059-0f347022da7c" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:18:01.901102 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.901078    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d76fb1a6-3241-475d-9059-0f347022da7c-config-out" (OuterVolumeSpecName: "config-out") pod "d76fb1a6-3241-475d-9059-0f347022da7c" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:18:01.901236 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.901204    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d76fb1a6-3241-475d-9059-0f347022da7c" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:01.901407 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.901374    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d76fb1a6-3241-475d-9059-0f347022da7c" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:01.901534 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.901421    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d76fb1a6-3241-475d-9059-0f347022da7c-kube-api-access-9j4z4" (OuterVolumeSpecName: "kube-api-access-9j4z4") pod "d76fb1a6-3241-475d-9059-0f347022da7c" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c"). InnerVolumeSpecName "kube-api-access-9j4z4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:18:01.901606 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.901583    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d76fb1a6-3241-475d-9059-0f347022da7c" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:01.901698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.901683    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d76fb1a6-3241-475d-9059-0f347022da7c" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:01.905831 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.905810    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d76fb1a6-3241-475d-9059-0f347022da7c" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:01.911657 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.911632    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-web-config" (OuterVolumeSpecName: "web-config") pod "d76fb1a6-3241-475d-9059-0f347022da7c" (UID: "d76fb1a6-3241-475d-9059-0f347022da7c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:01.999228 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.999180    2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-config-volume\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:01.999228 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.999221    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d76fb1a6-3241-475d-9059-0f347022da7c-tls-assets\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:01.999228 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.999230    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-web-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:01.999228 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.999239    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:01.999228 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.999248    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d76fb1a6-3241-475d-9059-0f347022da7c-config-out\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:01.999563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.999258    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d76fb1a6-3241-475d-9059-0f347022da7c-metrics-client-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:01.999563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.999266    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j4z4\" (UniqueName: \"kubernetes.io/projected/d76fb1a6-3241-475d-9059-0f347022da7c-kube-api-access-9j4z4\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:01.999563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.999275    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d76fb1a6-3241-475d-9059-0f347022da7c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:01.999563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.999283    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-main-tls\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:01.999563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.999291    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d76fb1a6-3241-475d-9059-0f347022da7c-alertmanager-main-db\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:01.999563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.999299    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:01.999563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.999307    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:01.999563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:01.999316    2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d76fb1a6-3241-475d-9059-0f347022da7c-cluster-tls-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:02.334735 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.334703    2560 generic.go:356] "Generic (PLEG): container finished" podID="d76fb1a6-3241-475d-9059-0f347022da7c" containerID="c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b" exitCode=0
Jul 15 19:18:02.334735 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.334728    2560 generic.go:356] "Generic (PLEG): container finished" podID="d76fb1a6-3241-475d-9059-0f347022da7c" containerID="a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f" exitCode=0
Jul 15 19:18:02.334952 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.334752    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerDied","Data":"c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b"}
Jul 15 19:18:02.334952 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.334790    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 15 19:18:02.334952 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.334806    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerDied","Data":"a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f"}
Jul 15 19:18:02.334952 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.334831    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d76fb1a6-3241-475d-9059-0f347022da7c","Type":"ContainerDied","Data":"c1f3abb774a3b6e8a4bfb6f90ab6a14a6b1055ed2e8921188bbd8cdc776beffa"}
Jul 15 19:18:02.334952 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.334868    2560 scope.go:117] "RemoveContainer" containerID="e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410"
Jul 15 19:18:02.342777 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.342543    2560 scope.go:117] "RemoveContainer" containerID="c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b"
Jul 15 19:18:02.349380 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.349360    2560 scope.go:117] "RemoveContainer" containerID="83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515"
Jul 15 19:18:02.356118 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.356099    2560 scope.go:117] "RemoveContainer" containerID="a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f"
Jul 15 19:18:02.357686 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.357441    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 19:18:02.361919 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.361896    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 15 19:18:02.364037 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.364023    2560 scope.go:117] "RemoveContainer" containerID="b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd"
Jul 15 19:18:02.370584 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.370567    2560 scope.go:117] "RemoveContainer" containerID="b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83"
Jul 15 19:18:02.377516 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.377493    2560 scope.go:117] "RemoveContainer" containerID="29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4"
Jul 15 19:18:02.384597 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.384509    2560 scope.go:117] "RemoveContainer" containerID="e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410"
Jul 15 19:18:02.384989 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:02.384967    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410\": container with ID starting with e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410 not found: ID does not exist" containerID="e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410"
Jul 15 19:18:02.385066 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.384999    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410"} err="failed to get container status \"e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410\": rpc error: code = NotFound desc = could not find container \"e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410\": container with ID starting with e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410 not found: ID does not exist"
Jul 15 19:18:02.385066 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.385033    2560 scope.go:117] "RemoveContainer" containerID="c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b"
Jul 15 19:18:02.385273 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:02.385257    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b\": container with ID starting with c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b not found: ID does not exist" containerID="c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b"
Jul 15 19:18:02.385306 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.385278    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b"} err="failed to get container status \"c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b\": rpc error: code = NotFound desc = could not find container \"c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b\": container with ID starting with c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b not found: ID does not exist"
Jul 15 19:18:02.385306 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.385297    2560 scope.go:117] "RemoveContainer" containerID="83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515"
Jul 15 19:18:02.385519 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:02.385504    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515\": container with ID starting with 83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515 not found: ID does not exist" containerID="83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515"
Jul 15 19:18:02.385598 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.385523    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515"} err="failed to get container status \"83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515\": rpc error: code = NotFound desc = could not find container \"83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515\": container with ID starting with 83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515 not found: ID does not exist"
Jul 15 19:18:02.385598 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.385544    2560 scope.go:117] "RemoveContainer" containerID="a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f"
Jul 15 19:18:02.385764 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:02.385745    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f\": container with ID starting with a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f not found: ID does not exist" containerID="a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f"
Jul 15 19:18:02.385798 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.385770    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f"} err="failed to get container status \"a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f\": rpc error: code = NotFound desc = could not find container \"a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f\": container with ID starting with a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f not found: ID does not exist"
Jul 15 19:18:02.385798 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.385787    2560 scope.go:117] "RemoveContainer" containerID="b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd"
Jul 15 19:18:02.385978 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:02.385965    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd\": container with ID starting with b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd not found: ID does not exist" containerID="b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd"
Jul 15 19:18:02.386013 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.385981    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd"} err="failed to get container status \"b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd\": rpc error: code = NotFound desc = could not find container \"b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd\": container with ID starting with b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd not found: ID does not exist"
Jul 15 19:18:02.386013 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.385992    2560 scope.go:117] "RemoveContainer" containerID="b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83"
Jul 15 19:18:02.386181 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:02.386161    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83\": container with ID starting with b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83 not found: ID does not exist" containerID="b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83"
Jul 15 19:18:02.386181 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.386177    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83"} err="failed to get container status \"b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83\": rpc error: code = NotFound desc = could not find container \"b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83\": container with ID starting with b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83 not found: ID does not exist"
Jul 15 19:18:02.386310 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.386189    2560 scope.go:117] "RemoveContainer" containerID="29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4"
Jul 15 19:18:02.386543 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:02.386523    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4\": container with ID starting with 29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4 not found: ID does not exist" containerID="29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4"
Jul 15 19:18:02.386637 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.386549    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4"} err="failed to get container status \"29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4\": rpc error: code = NotFound desc = could not find container \"29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4\": container with ID starting with 29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4 not found: ID does not exist"
Jul 15 19:18:02.386637 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.386563    2560 scope.go:117] "RemoveContainer" containerID="e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410"
Jul 15 19:18:02.386847 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.386803    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410"} err="failed to get container status \"e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410\": rpc error: code = NotFound desc = could not find container \"e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410\": container with ID starting with e7f77331688d9d816cbf2af48c65f910fa73d36b3ca9276af042acfbb10ab410 not found: ID does not exist"
Jul 15 19:18:02.386921 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.386851    2560 scope.go:117] "RemoveContainer" containerID="c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b"
Jul 15 19:18:02.387197 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.387174    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b"} err="failed to get container status \"c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b\": rpc error: code = NotFound desc = could not find container \"c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b\": container with ID starting with c85603b3eebe09fd8b2ace6734a078792308682c9ae4b6070483651d255ad24b not found: ID does not exist"
Jul 15 19:18:02.387261 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.387200    2560 scope.go:117] "RemoveContainer" containerID="83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515"
Jul 15 19:18:02.387435 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.387417    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515"} err="failed to get container status \"83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515\": rpc error: code = NotFound desc = could not find container \"83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515\": container with ID starting with 83b1f2020e9bdf869225f2dd8e64e8dae20a9f5163c74c76ce5a26e4f4d39515 not found: ID does not exist"
Jul 15 19:18:02.387498 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.387435    2560 scope.go:117] "RemoveContainer" containerID="a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f"
Jul 15 19:18:02.387689 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.387669    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f"} err="failed to get container status \"a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f\": rpc error: code = NotFound desc = could not find container \"a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f\": container with ID starting with a67590f7bce8aa0b8d550b1f797af2b5dd8d7bb0470e57eed6c6384950c4aa5f not found: ID does not exist"
Jul 15 19:18:02.387728 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.387692    2560 scope.go:117] "RemoveContainer" containerID="b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd"
Jul 15 19:18:02.387927 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.387905    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd"} err="failed to get container status \"b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd\": rpc error: code = NotFound desc = could not find container \"b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd\": container with ID starting with b933c26ac563b722e3ce5e33e245d662edb094b6fede8c53ed96fcfd050723dd not found: ID does not exist"
Jul 15 19:18:02.387968 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.387929    2560 scope.go:117] "RemoveContainer" containerID="b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83"
Jul 15 19:18:02.388207 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.388177    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83"} err="failed to get container status \"b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83\": rpc error: code = NotFound desc = could not find container \"b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83\": container with ID starting with b9b3366059933bee277950504cca9572ae4bef744155bd8e86fa1ed4ef120d83 not found: ID does not exist"
Jul 15 19:18:02.388308 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.388209    2560 scope.go:117] "RemoveContainer" containerID="29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4"
Jul 15 19:18:02.388499 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.388476    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4"} err="failed to get container status \"29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4\": rpc error: code = NotFound desc = could not find container \"29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4\": container with ID starting with 29c967e337024abe61280b0a0bd0d747d34443f573c6bfe2507fe24097950ec4 not found: ID does not exist"
Jul 15 19:18:02.494395 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:02.494364    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" path="/var/lib/kubelet/pods/d76fb1a6-3241-475d-9059-0f347022da7c/volumes"
Jul 15 19:18:04.853379 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:04.853341    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 19:18:04.853960 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:04.853736    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="prometheus" containerID="cri-o://487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31" gracePeriod=600
Jul 15 19:18:04.854146 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:04.854113    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="kube-rbac-proxy-web" containerID="cri-o://e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6" gracePeriod=600
Jul 15 19:18:04.854291 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:04.854258    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="kube-rbac-proxy" containerID="cri-o://cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e" gracePeriod=600
Jul 15 19:18:04.854393 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:04.854261    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="config-reloader" containerID="cri-o://ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e" gracePeriod=600
Jul 15 19:18:04.854505 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:04.854386    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="thanos-sidecar" containerID="cri-o://c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b" gracePeriod=600
Jul 15 19:18:04.854675 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:04.854507    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="kube-rbac-proxy-thanos" containerID="cri-o://289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef" gracePeriod=600
Jul 15 19:18:05.123819 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.123789    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.229858 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.229827    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-trusted-ca-bundle\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.230036 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.229869    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-1\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.230036 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.229900    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxf5j\" (UniqueName: \"kubernetes.io/projected/90459a65-95e1-4d04-b11a-cb616cf8f7c0-kube-api-access-zxf5j\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.230036 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.229930    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-db\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.230036 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.229958    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-kubelet-serving-ca-bundle\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.230248 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.230135    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/90459a65-95e1-4d04-b11a-cb616cf8f7c0-tls-assets\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.230248 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.230213    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:18:05.230353 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.230282    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:18:05.230405 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.230378    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:18:05.230481 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.230436    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-thanos-prometheus-http-client-file\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.230539 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.230518    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.230600 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.230585    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.230646 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.230614    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-tls\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.230690 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.230645    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-metrics-client-ca\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.230690 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.230673    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-grpc-tls\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.230947 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.230902    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-serving-certs-ca-bundle\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.231041 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.230946    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/90459a65-95e1-4d04-b11a-cb616cf8f7c0-config-out\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.231041 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.230990    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-kube-rbac-proxy\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.231041 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.231015    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-metrics-client-certs\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.231188 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.231065    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-0\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.231188 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.231104    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-web-config\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.231188 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.231137    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-config\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.231188 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.231163    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-2\") pod \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\" (UID: \"90459a65-95e1-4d04-b11a-cb616cf8f7c0\") "
Jul 15 19:18:05.231390 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.231188    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:18:05.231477 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.231429    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.231537 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.231478    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.231537 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.231494    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-db\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.231537 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.231509    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.231805 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.231779    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:18:05.231972 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.231945    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:18:05.232320 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.232296    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:18:05.232968 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.232927    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90459a65-95e1-4d04-b11a-cb616cf8f7c0-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:18:05.233224 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.233192    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90459a65-95e1-4d04-b11a-cb616cf8f7c0-kube-api-access-zxf5j" (OuterVolumeSpecName: "kube-api-access-zxf5j") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "kube-api-access-zxf5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:18:05.233304 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.233242    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:18:05.234404 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.234380    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-config" (OuterVolumeSpecName: "config") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:05.234575 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.234546    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:05.234689 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.234579    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90459a65-95e1-4d04-b11a-cb616cf8f7c0-config-out" (OuterVolumeSpecName: "config-out") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:18:05.234689 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.234639    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:05.234891 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.234868    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:05.234958 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.234942    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:05.235148 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.235132    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:05.235383 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.235363    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:05.235906 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.235882    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:05.243328 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.243308    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-web-config" (OuterVolumeSpecName: "web-config") pod "90459a65-95e1-4d04-b11a-cb616cf8f7c0" (UID: "90459a65-95e1-4d04-b11a-cb616cf8f7c0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:05.332118 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332063    2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-kube-rbac-proxy\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332118 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332112    2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-metrics-client-certs\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332118 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332125    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332140    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-web-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332154    2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332165    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332176    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxf5j\" (UniqueName: \"kubernetes.io/projected/90459a65-95e1-4d04-b11a-cb616cf8f7c0-kube-api-access-zxf5j\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332187    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/90459a65-95e1-4d04-b11a-cb616cf8f7c0-tls-assets\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332201    2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332214    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332226    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332240    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332252    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-metrics-client-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332263    2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/90459a65-95e1-4d04-b11a-cb616cf8f7c0-secret-grpc-tls\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332275    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90459a65-95e1-4d04-b11a-cb616cf8f7c0-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.332385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.332286    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/90459a65-95e1-4d04-b11a-cb616cf8f7c0-config-out\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:05.350869 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.350828    2560 generic.go:356] "Generic (PLEG): container finished" podID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerID="289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef" exitCode=0
Jul 15 19:18:05.350869 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.350851    2560 generic.go:356] "Generic (PLEG): container finished" podID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerID="cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e" exitCode=0
Jul 15 19:18:05.350869 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.350864    2560 generic.go:356] "Generic (PLEG): container finished" podID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerID="e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6" exitCode=0
Jul 15 19:18:05.350869 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.350870    2560 generic.go:356] "Generic (PLEG): container finished" podID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerID="c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b" exitCode=0
Jul 15 19:18:05.350869 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.350875    2560 generic.go:356] "Generic (PLEG): container finished" podID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerID="ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e" exitCode=0
Jul 15 19:18:05.350869 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.350880    2560 generic.go:356] "Generic (PLEG): container finished" podID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerID="487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31" exitCode=0
Jul 15 19:18:05.351271 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.350906    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerDied","Data":"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"}
Jul 15 19:18:05.351271 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.350933    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.351271 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.350977    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerDied","Data":"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"}
Jul 15 19:18:05.351271 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.351006    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerDied","Data":"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"}
Jul 15 19:18:05.351271 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.351034    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerDied","Data":"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"}
Jul 15 19:18:05.351271 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.351042    2560 scope.go:117] "RemoveContainer" containerID="289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"
Jul 15 19:18:05.351271 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.351058    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerDied","Data":"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"}
Jul 15 19:18:05.351271 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.351085    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerDied","Data":"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"}
Jul 15 19:18:05.351271 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.351109    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90459a65-95e1-4d04-b11a-cb616cf8f7c0","Type":"ContainerDied","Data":"22d007cc6eb006cc0734c5c6f34f34a83a12737aaf2cccf6c652cab6f1f26efe"}
Jul 15 19:18:05.359163 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.359142    2560 scope.go:117] "RemoveContainer" containerID="cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"
Jul 15 19:18:05.366150 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.366131    2560 scope.go:117] "RemoveContainer" containerID="e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"
Jul 15 19:18:05.373257 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.373233    2560 scope.go:117] "RemoveContainer" containerID="c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"
Jul 15 19:18:05.374337 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.374193    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 19:18:05.378498 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.378478    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 19:18:05.380519 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.380499    2560 scope.go:117] "RemoveContainer" containerID="ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"
Jul 15 19:18:05.386899 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.386883    2560 scope.go:117] "RemoveContainer" containerID="487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"
Jul 15 19:18:05.393592 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.393574    2560 scope.go:117] "RemoveContainer" containerID="b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"
Jul 15 19:18:05.402014 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.401991    2560 scope.go:117] "RemoveContainer" containerID="289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"
Jul 15 19:18:05.402287 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:05.402269    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef\": container with ID starting with 289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef not found: ID does not exist" containerID="289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"
Jul 15 19:18:05.402342 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.402302    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"} err="failed to get container status \"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef\": rpc error: code = NotFound desc = could not find container \"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef\": container with ID starting with 289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef not found: ID does not exist"
Jul 15 19:18:05.402342 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.402323    2560 scope.go:117] "RemoveContainer" containerID="cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"
Jul 15 19:18:05.402595 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:05.402576    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e\": container with ID starting with cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e not found: ID does not exist" containerID="cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"
Jul 15 19:18:05.402663 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.402605    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"} err="failed to get container status \"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e\": rpc error: code = NotFound desc = could not find container \"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e\": container with ID starting with cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e not found: ID does not exist"
Jul 15 19:18:05.402663 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.402629    2560 scope.go:117] "RemoveContainer" containerID="e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"
Jul 15 19:18:05.402892 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:05.402848    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6\": container with ID starting with e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6 not found: ID does not exist" containerID="e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"
Jul 15 19:18:05.402936 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.402896    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"} err="failed to get container status \"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6\": rpc error: code = NotFound desc = could not find container \"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6\": container with ID starting with e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6 not found: ID does not exist"
Jul 15 19:18:05.402936 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.402908    2560 scope.go:117] "RemoveContainer" containerID="c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"
Jul 15 19:18:05.403138 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:05.403123    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b\": container with ID starting with c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b not found: ID does not exist" containerID="c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"
Jul 15 19:18:05.403203 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.403145    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"} err="failed to get container status \"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b\": rpc error: code = NotFound desc = could not find container \"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b\": container with ID starting with c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b not found: ID does not exist"
Jul 15 19:18:05.403203 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.403167    2560 scope.go:117] "RemoveContainer" containerID="ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"
Jul 15 19:18:05.403396 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:05.403381    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e\": container with ID starting with ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e not found: ID does not exist" containerID="ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"
Jul 15 19:18:05.403478 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.403401    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"} err="failed to get container status \"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e\": rpc error: code = NotFound desc = could not find container \"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e\": container with ID starting with ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e not found: ID does not exist"
Jul 15 19:18:05.403478 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.403421    2560 scope.go:117] "RemoveContainer" containerID="487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"
Jul 15 19:18:05.404062 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:05.403922    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31\": container with ID starting with 487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31 not found: ID does not exist" containerID="487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"
Jul 15 19:18:05.404062 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.403954    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"} err="failed to get container status \"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31\": rpc error: code = NotFound desc = could not find container \"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31\": container with ID starting with 487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31 not found: ID does not exist"
Jul 15 19:18:05.404062 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.403974    2560 scope.go:117] "RemoveContainer" containerID="b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"
Jul 15 19:18:05.404297 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:05.404271    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b\": container with ID starting with b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b not found: ID does not exist" containerID="b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"
Jul 15 19:18:05.404376 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.404305    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"} err="failed to get container status \"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b\": rpc error: code = NotFound desc = could not find container \"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b\": container with ID starting with b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b not found: ID does not exist"
Jul 15 19:18:05.404376 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.404326    2560 scope.go:117] "RemoveContainer" containerID="289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"
Jul 15 19:18:05.405107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.404621    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"} err="failed to get container status \"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef\": rpc error: code = NotFound desc = could not find container \"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef\": container with ID starting with 289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef not found: ID does not exist"
Jul 15 19:18:05.405107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.404644    2560 scope.go:117] "RemoveContainer" containerID="cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"
Jul 15 19:18:05.405107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.404838    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"} err="failed to get container status \"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e\": rpc error: code = NotFound desc = could not find container \"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e\": container with ID starting with cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e not found: ID does not exist"
Jul 15 19:18:05.405107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.404856    2560 scope.go:117] "RemoveContainer" containerID="e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"
Jul 15 19:18:05.405107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.405037    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"} err="failed to get container status \"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6\": rpc error: code = NotFound desc = could not find container \"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6\": container with ID starting with e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6 not found: ID does not exist"
Jul 15 19:18:05.405107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.405057    2560 scope.go:117] "RemoveContainer" containerID="c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"
Jul 15 19:18:05.405446 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.405321    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"} err="failed to get container status \"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b\": rpc error: code = NotFound desc = could not find container \"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b\": container with ID starting with c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b not found: ID does not exist"
Jul 15 19:18:05.405446 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.405343    2560 scope.go:117] "RemoveContainer" containerID="ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"
Jul 15 19:18:05.405625 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.405604    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"} err="failed to get container status \"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e\": rpc error: code = NotFound desc = could not find container \"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e\": container with ID starting with ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e not found: ID does not exist"
Jul 15 19:18:05.405667 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.405627    2560 scope.go:117] "RemoveContainer" containerID="487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"
Jul 15 19:18:05.405868 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.405851    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"} err="failed to get container status \"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31\": rpc error: code = NotFound desc = could not find container \"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31\": container with ID starting with 487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31 not found: ID does not exist"
Jul 15 19:18:05.405927 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.405868    2560 scope.go:117] "RemoveContainer" containerID="b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"
Jul 15 19:18:05.405972 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.405961    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 19:18:05.406077 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406059    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"} err="failed to get container status \"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b\": rpc error: code = NotFound desc = could not find container \"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b\": container with ID starting with b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b not found: ID does not exist"
Jul 15 19:18:05.406112 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406078    2560 scope.go:117] "RemoveContainer" containerID="289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"
Jul 15 19:18:05.406255 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406239    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="config-reloader"
Jul 15 19:18:05.406305 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406258    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="config-reloader"
Jul 15 19:18:05.406305 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406262    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"} err="failed to get container status \"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef\": rpc error: code = NotFound desc = could not find container \"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef\": container with ID starting with 289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef not found: ID does not exist"
Jul 15 19:18:05.406305 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406272    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="kube-rbac-proxy-metric"
Jul 15 19:18:05.406305 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406275    2560 scope.go:117] "RemoveContainer" containerID="cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"
Jul 15 19:18:05.406305 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406278    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="kube-rbac-proxy-metric"
Jul 15 19:18:05.406305 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406288    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9356193f-ee01-4c43-8ea1-7550f4b082af" containerName="registry"
Jul 15 19:18:05.406305 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406293    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9356193f-ee01-4c43-8ea1-7550f4b082af" containerName="registry"
Jul 15 19:18:05.406305 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406301    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="config-reloader"
Jul 15 19:18:05.406305 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406306    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="config-reloader"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406313    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="kube-rbac-proxy-web"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406319    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="kube-rbac-proxy-web"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406325    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="kube-rbac-proxy"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406330    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="kube-rbac-proxy"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406340    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="init-config-reloader"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406348    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="init-config-reloader"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406358    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="init-config-reloader"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406363    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="init-config-reloader"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406369    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="alertmanager"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406375    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="alertmanager"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406383    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="prometheus"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406390    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="prometheus"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406400    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ac8b408-ead4-4902-b9f5-a665c2b35656" containerName="console"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406408    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ac8b408-ead4-4902-b9f5-a665c2b35656" containerName="console"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406418    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="kube-rbac-proxy-web"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406426    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="kube-rbac-proxy-web"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406439    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="518fd2e3-df4d-432f-a374-5b28ddfb77f0" containerName="console"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406460    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="518fd2e3-df4d-432f-a374-5b28ddfb77f0" containerName="console"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406478    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="thanos-sidecar"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406487    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="thanos-sidecar"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406496    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="kube-rbac-proxy"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406504    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="kube-rbac-proxy"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406514    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="kube-rbac-proxy-thanos"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406510    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"} err="failed to get container status \"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e\": rpc error: code = NotFound desc = could not find container \"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e\": container with ID starting with cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e not found: ID does not exist"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406533    2560 scope.go:117] "RemoveContainer" containerID="e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406522    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="kube-rbac-proxy-thanos"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406582    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="prom-label-proxy"
Jul 15 19:18:05.406648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406591    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="prom-label-proxy"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406654    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="kube-rbac-proxy-metric"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406663    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="518fd2e3-df4d-432f-a374-5b28ddfb77f0" containerName="console"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406669    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="prom-label-proxy"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406674    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="kube-rbac-proxy"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406680    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="kube-rbac-proxy-thanos"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406690    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="thanos-sidecar"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406698    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ac8b408-ead4-4902-b9f5-a665c2b35656" containerName="console"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406706    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="prometheus"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406713    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="kube-rbac-proxy-web"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406720    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="kube-rbac-proxy-web"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406727    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="alertmanager"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406734    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="kube-rbac-proxy"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406741    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d76fb1a6-3241-475d-9059-0f347022da7c" containerName="config-reloader"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406746    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9356193f-ee01-4c43-8ea1-7550f4b082af" containerName="registry"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406751    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" containerName="config-reloader"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406776    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"} err="failed to get container status \"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6\": rpc error: code = NotFound desc = could not find container \"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6\": container with ID starting with e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6 not found: ID does not exist"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.406793    2560 scope.go:117] "RemoveContainer" containerID="c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.407099    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"} err="failed to get container status \"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b\": rpc error: code = NotFound desc = could not find container \"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b\": container with ID starting with c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b not found: ID does not exist"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.407130    2560 scope.go:117] "RemoveContainer" containerID="ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.407527    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"} err="failed to get container status \"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e\": rpc error: code = NotFound desc = could not find container \"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e\": container with ID starting with ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e not found: ID does not exist"
Jul 15 19:18:05.407644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.407554    2560 scope.go:117] "RemoveContainer" containerID="487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"
Jul 15 19:18:05.408331 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.407785    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"} err="failed to get container status \"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31\": rpc error: code = NotFound desc = could not find container \"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31\": container with ID starting with 487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31 not found: ID does not exist"
Jul 15 19:18:05.408331 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.407802    2560 scope.go:117] "RemoveContainer" containerID="b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"
Jul 15 19:18:05.408331 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.407973    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"} err="failed to get container status \"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b\": rpc error: code = NotFound desc = could not find container \"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b\": container with ID starting with b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b not found: ID does not exist"
Jul 15 19:18:05.408331 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.407984    2560 scope.go:117] "RemoveContainer" containerID="289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"
Jul 15 19:18:05.408331 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.408144    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"} err="failed to get container status \"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef\": rpc error: code = NotFound desc = could not find container \"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef\": container with ID starting with 289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef not found: ID does not exist"
Jul 15 19:18:05.408331 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.408156    2560 scope.go:117] "RemoveContainer" containerID="cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"
Jul 15 19:18:05.408551 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.408355    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"} err="failed to get container status \"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e\": rpc error: code = NotFound desc = could not find container \"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e\": container with ID starting with cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e not found: ID does not exist"
Jul 15 19:18:05.408551 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.408367    2560 scope.go:117] "RemoveContainer" containerID="e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"
Jul 15 19:18:05.408612 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.408566    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"} err="failed to get container status \"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6\": rpc error: code = NotFound desc = could not find container \"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6\": container with ID starting with e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6 not found: ID does not exist"
Jul 15 19:18:05.408612 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.408578    2560 scope.go:117] "RemoveContainer" containerID="c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"
Jul 15 19:18:05.408779 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.408764    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"} err="failed to get container status \"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b\": rpc error: code = NotFound desc = could not find container \"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b\": container with ID starting with c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b not found: ID does not exist"
Jul 15 19:18:05.408818 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.408779    2560 scope.go:117] "RemoveContainer" containerID="ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"
Jul 15 19:18:05.408976 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.408959    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"} err="failed to get container status \"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e\": rpc error: code = NotFound desc = could not find container \"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e\": container with ID starting with ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e not found: ID does not exist"
Jul 15 19:18:05.409016 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.408975    2560 scope.go:117] "RemoveContainer" containerID="487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"
Jul 15 19:18:05.409146 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.409130    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"} err="failed to get container status \"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31\": rpc error: code = NotFound desc = could not find container \"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31\": container with ID starting with 487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31 not found: ID does not exist"
Jul 15 19:18:05.409187 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.409146    2560 scope.go:117] "RemoveContainer" containerID="b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"
Jul 15 19:18:05.409303 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.409289    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"} err="failed to get container status \"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b\": rpc error: code = NotFound desc = could not find container \"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b\": container with ID starting with b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b not found: ID does not exist"
Jul 15 19:18:05.409339 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.409303    2560 scope.go:117] "RemoveContainer" containerID="289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"
Jul 15 19:18:05.409521 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.409501    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"} err="failed to get container status \"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef\": rpc error: code = NotFound desc = could not find container \"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef\": container with ID starting with 289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef not found: ID does not exist"
Jul 15 19:18:05.409584 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.409521    2560 scope.go:117] "RemoveContainer" containerID="cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"
Jul 15 19:18:05.409857 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.409828    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"} err="failed to get container status \"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e\": rpc error: code = NotFound desc = could not find container \"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e\": container with ID starting with cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e not found: ID does not exist"
Jul 15 19:18:05.409857 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.409857    2560 scope.go:117] "RemoveContainer" containerID="e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"
Jul 15 19:18:05.410163 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.410139    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"} err="failed to get container status \"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6\": rpc error: code = NotFound desc = could not find container \"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6\": container with ID starting with e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6 not found: ID does not exist"
Jul 15 19:18:05.410163 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.410162    2560 scope.go:117] "RemoveContainer" containerID="c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"
Jul 15 19:18:05.410417 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.410400    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"} err="failed to get container status \"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b\": rpc error: code = NotFound desc = could not find container \"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b\": container with ID starting with c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b not found: ID does not exist"
Jul 15 19:18:05.410417 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.410418    2560 scope.go:117] "RemoveContainer" containerID="ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"
Jul 15 19:18:05.410671 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.410654    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"} err="failed to get container status \"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e\": rpc error: code = NotFound desc = could not find container \"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e\": container with ID starting with ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e not found: ID does not exist"
Jul 15 19:18:05.410671 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.410670    2560 scope.go:117] "RemoveContainer" containerID="487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"
Jul 15 19:18:05.410864 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.410839    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.410933 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.410887    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"} err="failed to get container status \"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31\": rpc error: code = NotFound desc = could not find container \"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31\": container with ID starting with 487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31 not found: ID does not exist"
Jul 15 19:18:05.410933 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.410904    2560 scope.go:117] "RemoveContainer" containerID="b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"
Jul 15 19:18:05.411159 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.411135    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"} err="failed to get container status \"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b\": rpc error: code = NotFound desc = could not find container \"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b\": container with ID starting with b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b not found: ID does not exist"
Jul 15 19:18:05.411249 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.411160    2560 scope.go:117] "RemoveContainer" containerID="289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"
Jul 15 19:18:05.411605 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.411558    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef"} err="failed to get container status \"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef\": rpc error: code = NotFound desc = could not find container \"289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef\": container with ID starting with 289d7a9509d543d5bcdeb57b663f1e7721543beef286ccfee2217226660983ef not found: ID does not exist"
Jul 15 19:18:05.411605 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.411590    2560 scope.go:117] "RemoveContainer" containerID="cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"
Jul 15 19:18:05.411835 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.411813    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e"} err="failed to get container status \"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e\": rpc error: code = NotFound desc = could not find container \"cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e\": container with ID starting with cda6eaef092a7fda8206cefbd30e821bafbb8d17ac99df8b2b434c25a19f0c2e not found: ID does not exist"
Jul 15 19:18:05.411835 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.411835    2560 scope.go:117] "RemoveContainer" containerID="e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"
Jul 15 19:18:05.412062 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.412044    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6"} err="failed to get container status \"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6\": rpc error: code = NotFound desc = could not find container \"e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6\": container with ID starting with e40132228aaa771a14f5d91596b8934f86a1c57612417315d419f7a21bef13a6 not found: ID does not exist"
Jul 15 19:18:05.412129 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.412063    2560 scope.go:117] "RemoveContainer" containerID="c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"
Jul 15 19:18:05.412299 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.412280    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b"} err="failed to get container status \"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b\": rpc error: code = NotFound desc = could not find container \"c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b\": container with ID starting with c0c972249cce679503f2c06e3e9c15c1913beb257feb9e30a9f1f3404845242b not found: ID does not exist"
Jul 15 19:18:05.412353 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.412300    2560 scope.go:117] "RemoveContainer" containerID="ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"
Jul 15 19:18:05.412525 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.412507    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e"} err="failed to get container status \"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e\": rpc error: code = NotFound desc = could not find container \"ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e\": container with ID starting with ddcd164415194632eaf22bbb3707ad28ea5d5ff0cbad9ac4795e415b66b0507e not found: ID does not exist"
Jul 15 19:18:05.412586 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.412526    2560 scope.go:117] "RemoveContainer" containerID="487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"
Jul 15 19:18:05.412702 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.412686    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31"} err="failed to get container status \"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31\": rpc error: code = NotFound desc = could not find container \"487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31\": container with ID starting with 487cdba3b38e1fede9cf0052a11b133a2c27c3317efdb76957f011a8e43b4b31 not found: ID does not exist"
Jul 15 19:18:05.412743 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.412702    2560 scope.go:117] "RemoveContainer" containerID="b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"
Jul 15 19:18:05.412897 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.412878    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b"} err="failed to get container status \"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b\": rpc error: code = NotFound desc = could not find container \"b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b\": container with ID starting with b258702f6a0358a93b81fc199481928550133afdfef93be8c7bc487d47f67f8b not found: ID does not exist"
Jul 15 19:18:05.413373 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.413358    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 19:18:05.413444 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.413372    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 19:18:05.413657 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.413642    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 19:18:05.413716 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.413707    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 19:18:05.413770 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.413716    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 19:18:05.413770 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.413711    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 19:18:05.414000 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.413987    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5rg9r\""
Jul 15 19:18:05.414391 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.414344    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 19:18:05.414560 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.414398    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 19:18:05.414560 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.414404    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 19:18:05.414560 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.414404    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 19:18:05.414560 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.414473    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3ntpsmmkrac0o\""
Jul 15 19:18:05.415016 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.414963    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 19:18:05.415016 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.414975    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 19:18:05.424170 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.420431    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 19:18:05.426507 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.426484    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 19:18:05.427468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.427431    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 19:18:05.534221 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534183    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534221 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534228    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534435 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534256    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khg8g\" (UniqueName: \"kubernetes.io/projected/7e473734-b386-4631-bb86-ef3e295238f9-kube-api-access-khg8g\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534435 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534332    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534435 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534379    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534435 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534411    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534435 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534431    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534627 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534479    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534627 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534498    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534627 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534515    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534627 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534543    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534627 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534565    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534627 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534581    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534627 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534599    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e473734-b386-4631-bb86-ef3e295238f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534627 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534620    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534848 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534659    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534848 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534686    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534848 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534717    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534848 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534745    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-config\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.534848 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.534764    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e473734-b386-4631-bb86-ef3e295238f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.635576 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635486    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.635576 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635529    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.635576 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635547    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.635576 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635567    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e473734-b386-4631-bb86-ef3e295238f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.635859 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635587    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.635859 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635658    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.635859 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635704    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.635859 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635737    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.635859 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635789    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-config\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.635859 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635839    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e473734-b386-4631-bb86-ef3e295238f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.636144 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635875    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.636144 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635911    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.636144 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635954    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khg8g\" (UniqueName: \"kubernetes.io/projected/7e473734-b386-4631-bb86-ef3e295238f9-kube-api-access-khg8g\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.636144 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.635983    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.636144 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.636028    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.636144 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.636067    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.636144 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.636097    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.636144 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.636131    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.636559 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.636157    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.636559 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.636187    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.636666 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.636596    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.636727 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.636704    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.636882 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.636861    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.637271 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.637248    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.637347 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.637306    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.638288 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.638255    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.638518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.638497    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e473734-b386-4631-bb86-ef3e295238f9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.638654 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.638624    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.638765 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.638686    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.638765 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.638734    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.640284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.639058    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.640284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.639172    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-config\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.640284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.639704    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.640617 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.640591    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.640776 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.640758    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.640887 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.640766    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.641035 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.641015    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e473734-b386-4631-bb86-ef3e295238f9-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.641155 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.641137    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e473734-b386-4631-bb86-ef3e295238f9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.642026 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.642004    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e473734-b386-4631-bb86-ef3e295238f9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.646316 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.646299    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khg8g\" (UniqueName: \"kubernetes.io/projected/7e473734-b386-4631-bb86-ef3e295238f9-kube-api-access-khg8g\") pod \"prometheus-k8s-0\" (UID: \"7e473734-b386-4631-bb86-ef3e295238f9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.724031 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.723995    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:05.852713 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:05.852689    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 19:18:05.854795 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:18:05.854766    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e473734_b386_4631_bb86_ef3e295238f9.slice/crio-17d6ab3f277b84df645ba4c618ef456da8ebe59cefedde439f45f8f832de6d22 WatchSource:0}: Error finding container 17d6ab3f277b84df645ba4c618ef456da8ebe59cefedde439f45f8f832de6d22: Status 404 returned error can't find the container with id 17d6ab3f277b84df645ba4c618ef456da8ebe59cefedde439f45f8f832de6d22
Jul 15 19:18:06.356195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:06.356161    2560 generic.go:356] "Generic (PLEG): container finished" podID="7e473734-b386-4631-bb86-ef3e295238f9" containerID="cafed42d086145af9826c27ec574d00336a9caf5305ad0be8dcfc827830a8578" exitCode=0
Jul 15 19:18:06.356363 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:06.356248    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e473734-b386-4631-bb86-ef3e295238f9","Type":"ContainerDied","Data":"cafed42d086145af9826c27ec574d00336a9caf5305ad0be8dcfc827830a8578"}
Jul 15 19:18:06.356363 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:06.356332    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e473734-b386-4631-bb86-ef3e295238f9","Type":"ContainerStarted","Data":"17d6ab3f277b84df645ba4c618ef456da8ebe59cefedde439f45f8f832de6d22"}
Jul 15 19:18:06.498577 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:06.498231    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90459a65-95e1-4d04-b11a-cb616cf8f7c0" path="/var/lib/kubelet/pods/90459a65-95e1-4d04-b11a-cb616cf8f7c0/volumes"
Jul 15 19:18:07.364717 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:07.364684    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e473734-b386-4631-bb86-ef3e295238f9","Type":"ContainerStarted","Data":"6a84cba3ef24886323bb90ef2abfe0133c9deb874b56e7dada24dbb25f3d2efd"}
Jul 15 19:18:07.365067 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:07.364748    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e473734-b386-4631-bb86-ef3e295238f9","Type":"ContainerStarted","Data":"09d13119cd57d850dfd893d70e2d38f7b3967dcfaa944bead4010861492660fa"}
Jul 15 19:18:07.365067 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:07.364774    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e473734-b386-4631-bb86-ef3e295238f9","Type":"ContainerStarted","Data":"2c83557d47a378af75d07afc2ab3a32671db90693334dbcccf4e3f16bd93971f"}
Jul 15 19:18:07.365067 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:07.364796    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e473734-b386-4631-bb86-ef3e295238f9","Type":"ContainerStarted","Data":"551a7d299984e4645ff4660996ea70ab790e75a84ebb28c731c0ed578c9e6d94"}
Jul 15 19:18:07.365067 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:07.364818    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e473734-b386-4631-bb86-ef3e295238f9","Type":"ContainerStarted","Data":"82599233c474e1cb35243ee2276a20d977344c5563561d69c9a1c559bc0e34cb"}
Jul 15 19:18:07.365067 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:07.364840    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e473734-b386-4631-bb86-ef3e295238f9","Type":"ContainerStarted","Data":"103907c2a28cc2a25e38d7156aab9d8978aa7c611339d594771f2626000524d0"}
Jul 15 19:18:07.394075 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:07.394022    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.372095276 podStartE2EDuration="2.394003393s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.908117ms" isStatefulPod=false podCreationTimestamp="2026-07-15 19:18:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:18:07.392540407 +0000 UTC m=+159.462399805" watchObservedRunningTime="2026-07-15 19:18:07.394003393 +0000 UTC m=+159.463862808"
Jul 15 19:18:08.090575 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.090535    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-767c8dc76-96zws"]
Jul 15 19:18:08.092937 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.092914    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.107210 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.107187    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-767c8dc76-96zws"]
Jul 15 19:18:08.144195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.144166    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:18:08.264435 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.264408    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-serving-cert\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.264575 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.264439    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsmx5\" (UniqueName: \"kubernetes.io/projected/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-kube-api-access-jsmx5\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.264575 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.264477    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-oauth-config\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.264575 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.264559    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-oauth-serving-cert\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.264691 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.264629    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-config\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.264691 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.264658    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-trusted-ca-bundle\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.264691 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.264674    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-service-ca\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.365192 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.365118    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-config\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.365192 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.365168    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-trusted-ca-bundle\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.365616 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.365309    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-service-ca\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.365616 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.365380    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-serving-cert\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.365616 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.365410    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsmx5\" (UniqueName: \"kubernetes.io/projected/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-kube-api-access-jsmx5\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.365616 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.365437    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-oauth-config\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.365616 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.365510    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-oauth-serving-cert\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.365922 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.365897    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-config\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.366033 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.366011    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-service-ca\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.366033 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.366023    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-trusted-ca-bundle\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.366215 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.366196    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-oauth-serving-cert\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.368070 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.368046    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-serving-cert\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.368491 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.368469    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-oauth-config\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.373400 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.373383    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsmx5\" (UniqueName: \"kubernetes.io/projected/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-kube-api-access-jsmx5\") pod \"console-767c8dc76-96zws\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") " pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.402419 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.402397    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:08.525784 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:08.525745    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-767c8dc76-96zws"]
Jul 15 19:18:08.527720 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:18:08.527696    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdd50274_2c33_49f5_b1e3_7db2c399cfc4.slice/crio-79c0bb492cf7af3aae5596847a352395e6a5413227c79b7faa449a143d2e1cb7 WatchSource:0}: Error finding container 79c0bb492cf7af3aae5596847a352395e6a5413227c79b7faa449a143d2e1cb7: Status 404 returned error can't find the container with id 79c0bb492cf7af3aae5596847a352395e6a5413227c79b7faa449a143d2e1cb7
Jul 15 19:18:09.145367 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:09.145333    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:18:09.380949 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:09.380912    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767c8dc76-96zws" event={"ID":"cdd50274-2c33-49f5-b1e3-7db2c399cfc4","Type":"ContainerStarted","Data":"748504ab26bbc614601d67eb64b66347bf70891a6e3ef0fad2736b7de5c9a904"}
Jul 15 19:18:09.381269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:09.380999    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767c8dc76-96zws" event={"ID":"cdd50274-2c33-49f5-b1e3-7db2c399cfc4","Type":"ContainerStarted","Data":"79c0bb492cf7af3aae5596847a352395e6a5413227c79b7faa449a143d2e1cb7"}
Jul 15 19:18:10.724833 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:10.724798    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:18:18.403206 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:18.403165    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:18.403608 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:18.403237    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:18.408144 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:18.408121    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:18.419000 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:18.418977    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:18:18.424468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:18.424411    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-767c8dc76-96zws" podStartSLOduration=10.424399402 podStartE2EDuration="10.424399402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:18:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:18:09.398013714 +0000 UTC m=+161.467873108" watchObservedRunningTime="2026-07-15 19:18:18.424399402 +0000 UTC m=+170.494258798"
Jul 15 19:18:18.460286 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:18.460254    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64d6fd76fc-jktfs"]
Jul 15 19:18:18.546291 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:18.546239    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:18:18.598654 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:18.598618    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:18:18.655346 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:18.655255    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:18:32.862651 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:32.862604    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:18:43.481260 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.481199    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64d6fd76fc-jktfs" podUID="bc05e11b-6928-4a9b-a085-90a1d3fcc7dd" containerName="console" containerID="cri-o://a6f6d6a0bfca8265c21f0afd5d8ebf95946cbe0e14d413cc37e306c21d01ea19" gracePeriod=15
Jul 15 19:18:43.750124 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.750096    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:18:43.858508 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.858474    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-oauth-config\") pod \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") "
Jul 15 19:18:43.858682 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.858547    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-trusted-ca-bundle\") pod \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") "
Jul 15 19:18:43.858682 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.858584    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-config\") pod \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") "
Jul 15 19:18:43.858682 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.858600    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-serving-cert\") pod \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") "
Jul 15 19:18:43.858682 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.858631    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-service-ca\") pod \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") "
Jul 15 19:18:43.858895 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.858723    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9fl7\" (UniqueName: \"kubernetes.io/projected/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-kube-api-access-h9fl7\") pod \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") "
Jul 15 19:18:43.858895 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.858766    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-oauth-serving-cert\") pod \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\" (UID: \"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd\") "
Jul 15 19:18:43.859032 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.859010    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-service-ca" (OuterVolumeSpecName: "service-ca") pod "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd" (UID: "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:18:43.859104 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.859021    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-config" (OuterVolumeSpecName: "console-config") pod "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd" (UID: "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:18:43.859104 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.859027    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd" (UID: "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:18:43.859283 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.859264    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd" (UID: "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:18:43.860642 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.860610    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd" (UID: "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:43.860745 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.860731    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd" (UID: "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:18:43.860785 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.860767    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-kube-api-access-h9fl7" (OuterVolumeSpecName: "kube-api-access-h9fl7") pod "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd" (UID: "bc05e11b-6928-4a9b-a085-90a1d3fcc7dd"). InnerVolumeSpecName "kube-api-access-h9fl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:18:43.960036 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.959989    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:43.960036 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.960031    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-serving-cert\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:43.960036 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.960041    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-service-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:43.960036 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.960050    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9fl7\" (UniqueName: \"kubernetes.io/projected/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-kube-api-access-h9fl7\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:43.960286 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.960058    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-oauth-serving-cert\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:43.960286 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.960066    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-console-oauth-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:43.960286 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:43.960074    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd-trusted-ca-bundle\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:18:44.519016 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:44.518984    2560 generic.go:356] "Generic (PLEG): container finished" podID="bc05e11b-6928-4a9b-a085-90a1d3fcc7dd" containerID="a6f6d6a0bfca8265c21f0afd5d8ebf95946cbe0e14d413cc37e306c21d01ea19" exitCode=0
Jul 15 19:18:44.519411 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:44.519034    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d6fd76fc-jktfs"
Jul 15 19:18:44.519411 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:44.519053    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d6fd76fc-jktfs" event={"ID":"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd","Type":"ContainerDied","Data":"a6f6d6a0bfca8265c21f0afd5d8ebf95946cbe0e14d413cc37e306c21d01ea19"}
Jul 15 19:18:44.519411 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:44.519092    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d6fd76fc-jktfs" event={"ID":"bc05e11b-6928-4a9b-a085-90a1d3fcc7dd","Type":"ContainerDied","Data":"19735d8162d1010dfc24bda91299aaddbe3e4f31c4e5620c24059dcb31e2c9ca"}
Jul 15 19:18:44.519411 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:44.519110    2560 scope.go:117] "RemoveContainer" containerID="a6f6d6a0bfca8265c21f0afd5d8ebf95946cbe0e14d413cc37e306c21d01ea19"
Jul 15 19:18:44.527051 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:44.527035    2560 scope.go:117] "RemoveContainer" containerID="a6f6d6a0bfca8265c21f0afd5d8ebf95946cbe0e14d413cc37e306c21d01ea19"
Jul 15 19:18:44.527301 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:18:44.527281    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6f6d6a0bfca8265c21f0afd5d8ebf95946cbe0e14d413cc37e306c21d01ea19\": container with ID starting with a6f6d6a0bfca8265c21f0afd5d8ebf95946cbe0e14d413cc37e306c21d01ea19 not found: ID does not exist" containerID="a6f6d6a0bfca8265c21f0afd5d8ebf95946cbe0e14d413cc37e306c21d01ea19"
Jul 15 19:18:44.527342 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:44.527308    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6f6d6a0bfca8265c21f0afd5d8ebf95946cbe0e14d413cc37e306c21d01ea19"} err="failed to get container status \"a6f6d6a0bfca8265c21f0afd5d8ebf95946cbe0e14d413cc37e306c21d01ea19\": rpc error: code = NotFound desc = could not find container \"a6f6d6a0bfca8265c21f0afd5d8ebf95946cbe0e14d413cc37e306c21d01ea19\": container with ID starting with a6f6d6a0bfca8265c21f0afd5d8ebf95946cbe0e14d413cc37e306c21d01ea19 not found: ID does not exist"
Jul 15 19:18:44.537495 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:44.537468    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64d6fd76fc-jktfs"]
Jul 15 19:18:44.540916 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:44.540893    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64d6fd76fc-jktfs"]
Jul 15 19:18:46.495209 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:46.495174    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc05e11b-6928-4a9b-a085-90a1d3fcc7dd" path="/var/lib/kubelet/pods/bc05e11b-6928-4a9b-a085-90a1d3fcc7dd/volumes"
Jul 15 19:18:48.424022 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:18:48.423977    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:19:00.853328 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:00.853266    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:19:05.724859 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:05.724811    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:19:05.739780 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:05.739756    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:19:06.614975 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:06.614946    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 19:19:07.033038 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:07.033001    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:19:14.782171 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:14.782131    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:19:14.830008 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:14.829980    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:19:26.692886 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:26.692848    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:19:26.744335 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:26.744294    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:19:29.181847 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.181817    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d9c556586-6v2lz"]
Jul 15 19:19:29.182182 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.182126    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc05e11b-6928-4a9b-a085-90a1d3fcc7dd" containerName="console"
Jul 15 19:19:29.182182 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.182139    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc05e11b-6928-4a9b-a085-90a1d3fcc7dd" containerName="console"
Jul 15 19:19:29.182251 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.182192    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc05e11b-6928-4a9b-a085-90a1d3fcc7dd" containerName="console"
Jul 15 19:19:29.185285 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.185265    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.195821 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.195801    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d9c556586-6v2lz"]
Jul 15 19:19:29.239465 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.239423    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-oauth-serving-cert\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.239585 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.239491    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-oauth-config\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.239585 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.239564    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-service-ca\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.239717 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.239608    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-config\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.239717 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.239643    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-serving-cert\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.239717 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.239672    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87w98\" (UniqueName: \"kubernetes.io/projected/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-kube-api-access-87w98\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.239717 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.239703    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-trusted-ca-bundle\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.270116 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.270093    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:19:29.340464 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.340424    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-config\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.340553 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.340470    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-serving-cert\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.340553 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.340489    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87w98\" (UniqueName: \"kubernetes.io/projected/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-kube-api-access-87w98\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.340635 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.340577    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-trusted-ca-bundle\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.340635 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.340597    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-oauth-serving-cert\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.340720 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.340636    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-oauth-config\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.340844 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.340800    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-service-ca\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.341374 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.341353    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-service-ca\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.341491 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.341413    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-oauth-serving-cert\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.341491 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.341476    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-trusted-ca-bundle\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.341491 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.341485    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-config\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.343027 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.343005    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-oauth-config\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.343141 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.343013    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-serving-cert\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.350481 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.350463    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87w98\" (UniqueName: \"kubernetes.io/projected/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-kube-api-access-87w98\") pod \"console-7d9c556586-6v2lz\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") " pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.370739 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.370712    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:19:29.493940 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.493913    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:29.614924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.614890    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d9c556586-6v2lz"]
Jul 15 19:19:29.617109 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:19:29.617083    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9dee4f2_cfa8_4c0e_91b2_b0eeff3b912f.slice/crio-d352aff755396a0e88ccb5461ff39eadbb49ac90553117c41b066bd8b2ce3ba1 WatchSource:0}: Error finding container d352aff755396a0e88ccb5461ff39eadbb49ac90553117c41b066bd8b2ce3ba1: Status 404 returned error can't find the container with id d352aff755396a0e88ccb5461ff39eadbb49ac90553117c41b066bd8b2ce3ba1
Jul 15 19:19:29.688399 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.688370    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d9c556586-6v2lz" event={"ID":"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f","Type":"ContainerStarted","Data":"9fb0e2ca33c1c343435089e4b5481564fe7bbd986a697331db55d876b9b5b24c"}
Jul 15 19:19:29.688514 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.688424    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d9c556586-6v2lz" event={"ID":"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f","Type":"ContainerStarted","Data":"d352aff755396a0e88ccb5461ff39eadbb49ac90553117c41b066bd8b2ce3ba1"}
Jul 15 19:19:29.704731 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:29.704683    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d9c556586-6v2lz" podStartSLOduration=0.704665127 podStartE2EDuration="704.665127ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:19:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:19:29.70412191 +0000 UTC m=+241.773981308" watchObservedRunningTime="2026-07-15 19:19:29.704665127 +0000 UTC m=+241.774524525"
Jul 15 19:19:31.331231 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:31.331164    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:19:31.432261 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:31.432231    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:19:39.494424 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:39.494386    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:39.494833 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:39.494491    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:39.498930 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:39.498902    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:39.731757 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:39.731727    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:19:39.773973 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:39.773893    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-767c8dc76-96zws"]
Jul 15 19:19:39.877844 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:39.877810    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:19:39.978755 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:39.978721    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:19:40.029838 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:19:40.029755    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:20:04.793870 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:04.793756    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-767c8dc76-96zws" podUID="cdd50274-2c33-49f5-b1e3-7db2c399cfc4" containerName="console" containerID="cri-o://748504ab26bbc614601d67eb64b66347bf70891a6e3ef0fad2736b7de5c9a904" gracePeriod=15
Jul 15 19:20:05.032644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.032621    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:20:05.133743 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.133675    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-trusted-ca-bundle\") pod \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") "
Jul 15 19:20:05.133743 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.133718    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-oauth-config\") pod \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") "
Jul 15 19:20:05.133743 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.133741    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsmx5\" (UniqueName: \"kubernetes.io/projected/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-kube-api-access-jsmx5\") pod \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") "
Jul 15 19:20:05.133997 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.133767    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-oauth-serving-cert\") pod \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") "
Jul 15 19:20:05.133997 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.133795    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-config\") pod \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") "
Jul 15 19:20:05.133997 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.133840    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-serving-cert\") pod \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") "
Jul 15 19:20:05.133997 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.133898    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-service-ca\") pod \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\" (UID: \"cdd50274-2c33-49f5-b1e3-7db2c399cfc4\") "
Jul 15 19:20:05.134206 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.134178    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cdd50274-2c33-49f5-b1e3-7db2c399cfc4" (UID: "cdd50274-2c33-49f5-b1e3-7db2c399cfc4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:20:05.134354 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.134329    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-config" (OuterVolumeSpecName: "console-config") pod "cdd50274-2c33-49f5-b1e3-7db2c399cfc4" (UID: "cdd50274-2c33-49f5-b1e3-7db2c399cfc4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:20:05.134354 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.134338    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cdd50274-2c33-49f5-b1e3-7db2c399cfc4" (UID: "cdd50274-2c33-49f5-b1e3-7db2c399cfc4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:20:05.134489 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.134357    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-service-ca" (OuterVolumeSpecName: "service-ca") pod "cdd50274-2c33-49f5-b1e3-7db2c399cfc4" (UID: "cdd50274-2c33-49f5-b1e3-7db2c399cfc4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:20:05.135916 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.135893    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-kube-api-access-jsmx5" (OuterVolumeSpecName: "kube-api-access-jsmx5") pod "cdd50274-2c33-49f5-b1e3-7db2c399cfc4" (UID: "cdd50274-2c33-49f5-b1e3-7db2c399cfc4"). InnerVolumeSpecName "kube-api-access-jsmx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:20:05.136291 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.136268    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cdd50274-2c33-49f5-b1e3-7db2c399cfc4" (UID: "cdd50274-2c33-49f5-b1e3-7db2c399cfc4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:20:05.136291 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.136284    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cdd50274-2c33-49f5-b1e3-7db2c399cfc4" (UID: "cdd50274-2c33-49f5-b1e3-7db2c399cfc4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:20:05.234764 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.234734    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-oauth-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:20:05.234764 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.234759    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsmx5\" (UniqueName: \"kubernetes.io/projected/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-kube-api-access-jsmx5\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:20:05.234764 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.234768    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-oauth-serving-cert\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:20:05.234985 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.234776    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:20:05.234985 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.234784    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-console-serving-cert\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:20:05.234985 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.234794    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-service-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:20:05.234985 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.234803    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdd50274-2c33-49f5-b1e3-7db2c399cfc4-trusted-ca-bundle\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:20:05.826659 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.826623    2560 generic.go:356] "Generic (PLEG): container finished" podID="cdd50274-2c33-49f5-b1e3-7db2c399cfc4" containerID="748504ab26bbc614601d67eb64b66347bf70891a6e3ef0fad2736b7de5c9a904" exitCode=0
Jul 15 19:20:05.827105 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.826664    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767c8dc76-96zws"
Jul 15 19:20:05.827105 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.826669    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767c8dc76-96zws" event={"ID":"cdd50274-2c33-49f5-b1e3-7db2c399cfc4","Type":"ContainerDied","Data":"748504ab26bbc614601d67eb64b66347bf70891a6e3ef0fad2736b7de5c9a904"}
Jul 15 19:20:05.827105 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.826768    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767c8dc76-96zws" event={"ID":"cdd50274-2c33-49f5-b1e3-7db2c399cfc4","Type":"ContainerDied","Data":"79c0bb492cf7af3aae5596847a352395e6a5413227c79b7faa449a143d2e1cb7"}
Jul 15 19:20:05.827105 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.826793    2560 scope.go:117] "RemoveContainer" containerID="748504ab26bbc614601d67eb64b66347bf70891a6e3ef0fad2736b7de5c9a904"
Jul 15 19:20:05.834713 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.834696    2560 scope.go:117] "RemoveContainer" containerID="748504ab26bbc614601d67eb64b66347bf70891a6e3ef0fad2736b7de5c9a904"
Jul 15 19:20:05.834956 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:20:05.834938    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"748504ab26bbc614601d67eb64b66347bf70891a6e3ef0fad2736b7de5c9a904\": container with ID starting with 748504ab26bbc614601d67eb64b66347bf70891a6e3ef0fad2736b7de5c9a904 not found: ID does not exist" containerID="748504ab26bbc614601d67eb64b66347bf70891a6e3ef0fad2736b7de5c9a904"
Jul 15 19:20:05.835005 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.834964    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"748504ab26bbc614601d67eb64b66347bf70891a6e3ef0fad2736b7de5c9a904"} err="failed to get container status \"748504ab26bbc614601d67eb64b66347bf70891a6e3ef0fad2736b7de5c9a904\": rpc error: code = NotFound desc = could not find container \"748504ab26bbc614601d67eb64b66347bf70891a6e3ef0fad2736b7de5c9a904\": container with ID starting with 748504ab26bbc614601d67eb64b66347bf70891a6e3ef0fad2736b7de5c9a904 not found: ID does not exist"
Jul 15 19:20:05.848781 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.848757    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-767c8dc76-96zws"]
Jul 15 19:20:05.853234 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:05.853215    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-767c8dc76-96zws"]
Jul 15 19:20:06.493862 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:06.493824    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdd50274-2c33-49f5-b1e3-7db2c399cfc4" path="/var/lib/kubelet/pods/cdd50274-2c33-49f5-b1e3-7db2c399cfc4/volumes"
Jul 15 19:20:10.141577 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:10.141544    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:20:11.359615 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:11.359583    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:20:28.346881 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:28.346843    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:20:28.351428 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:28.351406    2560 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 19:20:28.361309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:28.361291    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:20:44.961202 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:44.961171    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:20:45.057865 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:20:45.057830    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:11.604935 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:11.604901    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:12.385853 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:12.385793    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:15.202361 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:15.202323    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:15.303028 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:15.302993    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:16.284442 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:16.284408    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:16.423832 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:16.423799    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:17.416024 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:17.415991    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:17.513617 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:17.513586    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:19.633942 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:19.633905    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:19.858980 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:19.858943    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:22.395994 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:22.395914    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:22.451402 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:22.451370    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:27.834215 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:27.834181    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:34.675587 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:34.675552    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:34.732815 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:34.732779    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:38.413469 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:38.413421    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:38.512508 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:38.512477    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:42.738154 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:42.738120    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:42.838996 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:42.838965    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:48.443436 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:48.443399    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:21:48.543336 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:21:48.543307    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:00.776603 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:00.776570    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:00.877245 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:00.877214    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:02.572228 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:02.572192    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:06.702182 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.702156    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:06.754637 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.754610    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"]
Jul 15 19:22:06.754909 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.754896    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd50274-2c33-49f5-b1e3-7db2c399cfc4" containerName="console"
Jul 15 19:22:06.754957 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.754910    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd50274-2c33-49f5-b1e3-7db2c399cfc4" containerName="console"
Jul 15 19:22:06.754991 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.754974    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd50274-2c33-49f5-b1e3-7db2c399cfc4" containerName="console"
Jul 15 19:22:06.757694 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.757679    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:06.760761 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.760735    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 15 19:22:06.761001 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.760988    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 15 19:22:06.762068 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.762054    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 15 19:22:06.762163 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.762068    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 15 19:22:06.762163 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.762083    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8b9nr\""
Jul 15 19:22:06.762274 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.762184    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 15 19:22:06.771656 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.771636    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"]
Jul 15 19:22:06.792204 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.792183    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:06.872464 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.872429    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn5sk\" (UniqueName: \"kubernetes.io/projected/c5afab62-c61d-42c7-9a2c-ca77732acc8e-kube-api-access-dn5sk\") pod \"lws-controller-manager-6c74497d4c-6fq5b\" (UID: \"c5afab62-c61d-42c7-9a2c-ca77732acc8e\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:06.872551 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.872496    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5afab62-c61d-42c7-9a2c-ca77732acc8e-cert\") pod \"lws-controller-manager-6c74497d4c-6fq5b\" (UID: \"c5afab62-c61d-42c7-9a2c-ca77732acc8e\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:06.872551 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.872541    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5afab62-c61d-42c7-9a2c-ca77732acc8e-metrics-cert\") pod \"lws-controller-manager-6c74497d4c-6fq5b\" (UID: \"c5afab62-c61d-42c7-9a2c-ca77732acc8e\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:06.872663 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.872569    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c5afab62-c61d-42c7-9a2c-ca77732acc8e-manager-config\") pod \"lws-controller-manager-6c74497d4c-6fq5b\" (UID: \"c5afab62-c61d-42c7-9a2c-ca77732acc8e\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:06.973313 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.973264    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5afab62-c61d-42c7-9a2c-ca77732acc8e-cert\") pod \"lws-controller-manager-6c74497d4c-6fq5b\" (UID: \"c5afab62-c61d-42c7-9a2c-ca77732acc8e\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:06.973313 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.973301    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5afab62-c61d-42c7-9a2c-ca77732acc8e-metrics-cert\") pod \"lws-controller-manager-6c74497d4c-6fq5b\" (UID: \"c5afab62-c61d-42c7-9a2c-ca77732acc8e\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:06.973421 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.973327    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c5afab62-c61d-42c7-9a2c-ca77732acc8e-manager-config\") pod \"lws-controller-manager-6c74497d4c-6fq5b\" (UID: \"c5afab62-c61d-42c7-9a2c-ca77732acc8e\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:06.973421 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.973359    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn5sk\" (UniqueName: \"kubernetes.io/projected/c5afab62-c61d-42c7-9a2c-ca77732acc8e-kube-api-access-dn5sk\") pod \"lws-controller-manager-6c74497d4c-6fq5b\" (UID: \"c5afab62-c61d-42c7-9a2c-ca77732acc8e\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:06.974008 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.973984    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c5afab62-c61d-42c7-9a2c-ca77732acc8e-manager-config\") pod \"lws-controller-manager-6c74497d4c-6fq5b\" (UID: \"c5afab62-c61d-42c7-9a2c-ca77732acc8e\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:06.975622 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.975602    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5afab62-c61d-42c7-9a2c-ca77732acc8e-metrics-cert\") pod \"lws-controller-manager-6c74497d4c-6fq5b\" (UID: \"c5afab62-c61d-42c7-9a2c-ca77732acc8e\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:06.975707 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.975633    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5afab62-c61d-42c7-9a2c-ca77732acc8e-cert\") pod \"lws-controller-manager-6c74497d4c-6fq5b\" (UID: \"c5afab62-c61d-42c7-9a2c-ca77732acc8e\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:06.982019 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:06.982001    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn5sk\" (UniqueName: \"kubernetes.io/projected/c5afab62-c61d-42c7-9a2c-ca77732acc8e-kube-api-access-dn5sk\") pod \"lws-controller-manager-6c74497d4c-6fq5b\" (UID: \"c5afab62-c61d-42c7-9a2c-ca77732acc8e\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:07.066539 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:07.066517    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:07.189999 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:07.189978    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"]
Jul 15 19:22:07.192522 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:22:07.192487    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5afab62_c61d_42c7_9a2c_ca77732acc8e.slice/crio-259be02f4ed9f0646d55e4d2105b388b2809c60ef1f47871f225cba40c6be17d WatchSource:0}: Error finding container 259be02f4ed9f0646d55e4d2105b388b2809c60ef1f47871f225cba40c6be17d: Status 404 returned error can't find the container with id 259be02f4ed9f0646d55e4d2105b388b2809c60ef1f47871f225cba40c6be17d
Jul 15 19:22:07.194442 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:07.194425    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 19:22:07.288013 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:07.287989    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b" event={"ID":"c5afab62-c61d-42c7-9a2c-ca77732acc8e","Type":"ContainerStarted","Data":"259be02f4ed9f0646d55e4d2105b388b2809c60ef1f47871f225cba40c6be17d"}
Jul 15 19:22:10.302381 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:10.302347    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b" event={"ID":"c5afab62-c61d-42c7-9a2c-ca77732acc8e","Type":"ContainerStarted","Data":"9130363e58b03b9b8a808f52fd35cffa37d4d528ccdbfe9a7d823c3ac4af47f7"}
Jul 15 19:22:10.302763 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:10.302488    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:10.320213 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:10.320174    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b" podStartSLOduration=2.034800994 podStartE2EDuration="4.320161172s" totalImagesPullingTime="2.285360178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:22:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:10.3186618 +0000 UTC m=+402.388521199" watchObservedRunningTime="2026-07-15 19:22:10.320161172 +0000 UTC m=+402.390020567"
Jul 15 19:22:17.461271 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:17.461239    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:17.551079 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:17.551047    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:18.119996 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.119964    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:18.351114 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.351083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:18.457578 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.457474    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:18.479931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.479902    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"]
Jul 15 19:22:18.483394 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.483375    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.488549 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.488527    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 19:22:18.488677 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.488593    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 19:22:18.488677 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.488638    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-glh4z\""
Jul 15 19:22:18.488677 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.488638    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 19:22:18.488842 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.488760    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 19:22:18.500091 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.500069    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"]
Jul 15 19:22:18.569867 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.569841    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e411a261-a349-4b1c-9f8a-0269bef246b6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.570031 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.569877    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e411a261-a349-4b1c-9f8a-0269bef246b6-webhook-cert\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.570031 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.569901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsms7\" (UniqueName: \"kubernetes.io/projected/e411a261-a349-4b1c-9f8a-0269bef246b6-kube-api-access-hsms7\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.570031 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.569992    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e411a261-a349-4b1c-9f8a-0269bef246b6-tmp\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.570172 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.570036    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e411a261-a349-4b1c-9f8a-0269bef246b6-manifests\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.600155 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.600126    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:18.671153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.671120    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e411a261-a349-4b1c-9f8a-0269bef246b6-tmp\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.671517 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.671171    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e411a261-a349-4b1c-9f8a-0269bef246b6-manifests\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.671517 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.671217    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e411a261-a349-4b1c-9f8a-0269bef246b6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.671517 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.671235    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e411a261-a349-4b1c-9f8a-0269bef246b6-webhook-cert\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.671517 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.671271    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsms7\" (UniqueName: \"kubernetes.io/projected/e411a261-a349-4b1c-9f8a-0269bef246b6-kube-api-access-hsms7\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.671757 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.671713    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e411a261-a349-4b1c-9f8a-0269bef246b6-manifests\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.673319 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.673288    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e411a261-a349-4b1c-9f8a-0269bef246b6-tmp\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.673685 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.673661    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e411a261-a349-4b1c-9f8a-0269bef246b6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.673741 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.673706    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e411a261-a349-4b1c-9f8a-0269bef246b6-webhook-cert\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.679686 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.679656    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsms7\" (UniqueName: \"kubernetes.io/projected/e411a261-a349-4b1c-9f8a-0269bef246b6-kube-api-access-hsms7\") pod \"opendatahub-operator-controller-manager-f46f5478c-m22fw\" (UID: \"e411a261-a349-4b1c-9f8a-0269bef246b6\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.710176 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.710104    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:18.793623 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.793597    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:18.823515 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.823489    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:18.912071 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.912036    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:18.925742 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:18.925723    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"]
Jul 15 19:22:18.927834 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:22:18.927810    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode411a261_a349_4b1c_9f8a_0269bef246b6.slice/crio-8bb45544b28110f5c8d6ff98d38520c4029aa7be92ce9ed7680dac97e311c415 WatchSource:0}: Error finding container 8bb45544b28110f5c8d6ff98d38520c4029aa7be92ce9ed7680dac97e311c415: Status 404 returned error can't find the container with id 8bb45544b28110f5c8d6ff98d38520c4029aa7be92ce9ed7680dac97e311c415
Jul 15 19:22:19.009828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:19.009796    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:19.111622 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:19.111590    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:19.341047 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:19.340953    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw" event={"ID":"e411a261-a349-4b1c-9f8a-0269bef246b6","Type":"ContainerStarted","Data":"8bb45544b28110f5c8d6ff98d38520c4029aa7be92ce9ed7680dac97e311c415"}
Jul 15 19:22:21.310085 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:21.310047    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-6fq5b"
Jul 15 19:22:22.356519 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:22.356486    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw" event={"ID":"e411a261-a349-4b1c-9f8a-0269bef246b6","Type":"ContainerStarted","Data":"6c278363cdebf93f65285e2fae8e3b4885e78de4821aa5e012a91d7ec266ce48"}
Jul 15 19:22:23.362284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:23.362188    2560 generic.go:356] "Generic (PLEG): container finished" podID="e411a261-a349-4b1c-9f8a-0269bef246b6" containerID="6c278363cdebf93f65285e2fae8e3b4885e78de4821aa5e012a91d7ec266ce48" exitCode=0
Jul 15 19:22:23.362284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:23.362227    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw" event={"ID":"e411a261-a349-4b1c-9f8a-0269bef246b6","Type":"ContainerDied","Data":"6c278363cdebf93f65285e2fae8e3b4885e78de4821aa5e012a91d7ec266ce48"}
Jul 15 19:22:24.369287 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:24.369253    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw" event={"ID":"e411a261-a349-4b1c-9f8a-0269bef246b6","Type":"ContainerStarted","Data":"b1b6f02e43ac34c76db6e576e5cc34148028e29d5c863beb1c290599144cf0cf"}
Jul 15 19:22:24.369753 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:24.369390    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:24.393085 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:24.393028    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw" podStartSLOduration=1.9775254279999999 podStartE2EDuration="6.393011479s" totalImagesPullingTime="3.362053345s" totalInitContainerRuntime="1.053432706s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:22:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:24.39281084 +0000 UTC m=+416.462670237" watchObservedRunningTime="2026-07-15 19:22:24.393011479 +0000 UTC m=+416.462870876"
Jul 15 19:22:35.376526 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:35.376499    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-m22fw"
Jul 15 19:22:37.355245 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:37.355209    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:37.502504 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:37.502407    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:38.008512 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.008480    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mm4c8"]
Jul 15 19:22:38.022554 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.022528    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8"
Jul 15 19:22:38.025666 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.025640    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-82pcm\""
Jul 15 19:22:38.025948 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.025898    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 15 19:22:38.029170 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.029147    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mm4c8"]
Jul 15 19:22:38.099030 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.098998    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:38.129908 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.129881    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a570eb6-6ef3-49c2-9f97-620668fcef9e-cert\") pod \"odh-model-controller-7998c67dc4-mm4c8\" (UID: \"5a570eb6-6ef3-49c2-9f97-620668fcef9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8"
Jul 15 19:22:38.130033 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.129936    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdr52\" (UniqueName: \"kubernetes.io/projected/5a570eb6-6ef3-49c2-9f97-620668fcef9e-kube-api-access-wdr52\") pod \"odh-model-controller-7998c67dc4-mm4c8\" (UID: \"5a570eb6-6ef3-49c2-9f97-620668fcef9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8"
Jul 15 19:22:38.196532 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.196504    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:38.231379 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.231357    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a570eb6-6ef3-49c2-9f97-620668fcef9e-cert\") pod \"odh-model-controller-7998c67dc4-mm4c8\" (UID: \"5a570eb6-6ef3-49c2-9f97-620668fcef9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8"
Jul 15 19:22:38.231520 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.231399    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdr52\" (UniqueName: \"kubernetes.io/projected/5a570eb6-6ef3-49c2-9f97-620668fcef9e-kube-api-access-wdr52\") pod \"odh-model-controller-7998c67dc4-mm4c8\" (UID: \"5a570eb6-6ef3-49c2-9f97-620668fcef9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8"
Jul 15 19:22:38.231560 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:22:38.231520    2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 15 19:22:38.231593 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:22:38.231574    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a570eb6-6ef3-49c2-9f97-620668fcef9e-cert podName:5a570eb6-6ef3-49c2-9f97-620668fcef9e nodeName:}" failed. No retries permitted until 2026-07-15 19:22:38.731558757 +0000 UTC m=+430.801418136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5a570eb6-6ef3-49c2-9f97-620668fcef9e-cert") pod "odh-model-controller-7998c67dc4-mm4c8" (UID: "5a570eb6-6ef3-49c2-9f97-620668fcef9e") : secret "odh-model-controller-webhook-cert" not found
Jul 15 19:22:38.240529 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.240506    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdr52\" (UniqueName: \"kubernetes.io/projected/5a570eb6-6ef3-49c2-9f97-620668fcef9e-kube-api-access-wdr52\") pod \"odh-model-controller-7998c67dc4-mm4c8\" (UID: \"5a570eb6-6ef3-49c2-9f97-620668fcef9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8"
Jul 15 19:22:38.297316 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.297247    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:38.399499 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.399473    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:38.634849 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.634760    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-78lfd"]
Jul 15 19:22:38.637393 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.637367    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-78lfd"
Jul 15 19:22:38.639920 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.639901    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-lgxcp\""
Jul 15 19:22:38.640068 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.640057    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 15 19:22:38.649513 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.649492    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-78lfd"]
Jul 15 19:22:38.661486 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.661463    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:38.735546 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.735521    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a570eb6-6ef3-49c2-9f97-620668fcef9e-cert\") pod \"odh-model-controller-7998c67dc4-mm4c8\" (UID: \"5a570eb6-6ef3-49c2-9f97-620668fcef9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8"
Jul 15 19:22:38.737798 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.737779    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a570eb6-6ef3-49c2-9f97-620668fcef9e-cert\") pod \"odh-model-controller-7998c67dc4-mm4c8\" (UID: \"5a570eb6-6ef3-49c2-9f97-620668fcef9e\") " pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8"
Jul 15 19:22:38.836403 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.836378    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fsrn\" (UniqueName: \"kubernetes.io/projected/230e7abb-5a9a-422e-9302-d9d2091ee70a-kube-api-access-5fsrn\") pod \"model-serving-api-54d89bd5fc-78lfd\" (UID: \"230e7abb-5a9a-422e-9302-d9d2091ee70a\") " pod="opendatahub/model-serving-api-54d89bd5fc-78lfd"
Jul 15 19:22:38.836516 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.836411    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/230e7abb-5a9a-422e-9302-d9d2091ee70a-tls-certs\") pod \"model-serving-api-54d89bd5fc-78lfd\" (UID: \"230e7abb-5a9a-422e-9302-d9d2091ee70a\") " pod="opendatahub/model-serving-api-54d89bd5fc-78lfd"
Jul 15 19:22:38.933478 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.933393    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8"
Jul 15 19:22:38.937058 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.937036    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fsrn\" (UniqueName: \"kubernetes.io/projected/230e7abb-5a9a-422e-9302-d9d2091ee70a-kube-api-access-5fsrn\") pod \"model-serving-api-54d89bd5fc-78lfd\" (UID: \"230e7abb-5a9a-422e-9302-d9d2091ee70a\") " pod="opendatahub/model-serving-api-54d89bd5fc-78lfd"
Jul 15 19:22:38.937115 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.937081    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/230e7abb-5a9a-422e-9302-d9d2091ee70a-tls-certs\") pod \"model-serving-api-54d89bd5fc-78lfd\" (UID: \"230e7abb-5a9a-422e-9302-d9d2091ee70a\") " pod="opendatahub/model-serving-api-54d89bd5fc-78lfd"
Jul 15 19:22:38.937236 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:22:38.937220    2560 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 15 19:22:38.937299 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:22:38.937289    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/230e7abb-5a9a-422e-9302-d9d2091ee70a-tls-certs podName:230e7abb-5a9a-422e-9302-d9d2091ee70a nodeName:}" failed. No retries permitted until 2026-07-15 19:22:39.43726563 +0000 UTC m=+431.507125018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/230e7abb-5a9a-422e-9302-d9d2091ee70a-tls-certs") pod "model-serving-api-54d89bd5fc-78lfd" (UID: "230e7abb-5a9a-422e-9302-d9d2091ee70a") : secret "model-serving-api-tls" not found
Jul 15 19:22:38.949150 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:38.949119    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fsrn\" (UniqueName: \"kubernetes.io/projected/230e7abb-5a9a-422e-9302-d9d2091ee70a-kube-api-access-5fsrn\") pod \"model-serving-api-54d89bd5fc-78lfd\" (UID: \"230e7abb-5a9a-422e-9302-d9d2091ee70a\") " pod="opendatahub/model-serving-api-54d89bd5fc-78lfd"
Jul 15 19:22:39.259565 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:39.259539    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mm4c8"]
Jul 15 19:22:39.262272 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:22:39.262246    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a570eb6_6ef3_49c2_9f97_620668fcef9e.slice/crio-3cd283f664295f5f8f3c343c4d177995c2d68026fe7d11f8eaf6ae2bfcdd91d7 WatchSource:0}: Error finding container 3cd283f664295f5f8f3c343c4d177995c2d68026fe7d11f8eaf6ae2bfcdd91d7: Status 404 returned error can't find the container with id 3cd283f664295f5f8f3c343c4d177995c2d68026fe7d11f8eaf6ae2bfcdd91d7
Jul 15 19:22:39.306987 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:39.306966    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:39.431354 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:39.431320    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8" event={"ID":"5a570eb6-6ef3-49c2-9f97-620668fcef9e","Type":"ContainerStarted","Data":"3cd283f664295f5f8f3c343c4d177995c2d68026fe7d11f8eaf6ae2bfcdd91d7"}
Jul 15 19:22:39.440793 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:39.440772    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/230e7abb-5a9a-422e-9302-d9d2091ee70a-tls-certs\") pod \"model-serving-api-54d89bd5fc-78lfd\" (UID: \"230e7abb-5a9a-422e-9302-d9d2091ee70a\") " pod="opendatahub/model-serving-api-54d89bd5fc-78lfd"
Jul 15 19:22:39.442998 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:39.442972    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/230e7abb-5a9a-422e-9302-d9d2091ee70a-tls-certs\") pod \"model-serving-api-54d89bd5fc-78lfd\" (UID: \"230e7abb-5a9a-422e-9302-d9d2091ee70a\") " pod="opendatahub/model-serving-api-54d89bd5fc-78lfd"
Jul 15 19:22:39.547597 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:39.547521    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-78lfd"
Jul 15 19:22:39.683801 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:39.683771    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-78lfd"]
Jul 15 19:22:39.686916 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:22:39.686877    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod230e7abb_5a9a_422e_9302_d9d2091ee70a.slice/crio-db7c5a8b5e07d62f7bc1f7eb1fa95077aaeab53e1050a48dcfd609ee597377d5 WatchSource:0}: Error finding container db7c5a8b5e07d62f7bc1f7eb1fa95077aaeab53e1050a48dcfd609ee597377d5: Status 404 returned error can't find the container with id db7c5a8b5e07d62f7bc1f7eb1fa95077aaeab53e1050a48dcfd609ee597377d5
Jul 15 19:22:40.438952 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:40.438872    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-78lfd" event={"ID":"230e7abb-5a9a-422e-9302-d9d2091ee70a","Type":"ContainerStarted","Data":"db7c5a8b5e07d62f7bc1f7eb1fa95077aaeab53e1050a48dcfd609ee597377d5"}
Jul 15 19:22:42.347041 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:42.347000    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:42.397953 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:42.397915    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:43.456165 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:43.456124    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-78lfd" event={"ID":"230e7abb-5a9a-422e-9302-d9d2091ee70a","Type":"ContainerStarted","Data":"2199ab42b972ac3f05e29e5bfbc4ea6767becc0b227583723104b63b6ccf3e64"}
Jul 15 19:22:43.456616 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:43.456222    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-78lfd"
Jul 15 19:22:43.457554 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:43.457530    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8" event={"ID":"5a570eb6-6ef3-49c2-9f97-620668fcef9e","Type":"ContainerStarted","Data":"fdb51aa94dd06d987cdd239c609c9a83a8572aeacae3b9d4db4bdc015fdb0360"}
Jul 15 19:22:43.457668 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:43.457645    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8"
Jul 15 19:22:43.492072 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:43.492028    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-78lfd" podStartSLOduration=2.653058031 podStartE2EDuration="5.492014873s" totalImagesPullingTime="2.838956842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:22:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:43.490092567 +0000 UTC m=+435.559951966" watchObservedRunningTime="2026-07-15 19:22:43.492014873 +0000 UTC m=+435.561874270"
Jul 15 19:22:43.520842 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:43.520798    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8" podStartSLOduration=3.253659622 podStartE2EDuration="6.520783281s" totalImagesPullingTime="3.267123659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:43.520172314 +0000 UTC m=+435.590031721" watchObservedRunningTime="2026-07-15 19:22:43.520783281 +0000 UTC m=+435.590642677"
Jul 15 19:22:46.963018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:46.962986    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:47.088815 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.088786    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:47.088994 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.088975    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"]
Jul 15 19:22:47.091675 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.091659    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"
Jul 15 19:22:47.096028 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.096007    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 15 19:22:47.097045 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.097025    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-w7xzt\""
Jul 15 19:22:47.106979 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.106956    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acf759f2-50d0-45f3-a138-d8919ea53d58-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-s8vxc\" (UID: \"acf759f2-50d0-45f3-a138-d8919ea53d58\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"
Jul 15 19:22:47.107072 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.107016    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp4ph\" (UniqueName: \"kubernetes.io/projected/acf759f2-50d0-45f3-a138-d8919ea53d58-kube-api-access-fp4ph\") pod \"llmisvc-controller-manager-777c6bd7d5-s8vxc\" (UID: \"acf759f2-50d0-45f3-a138-d8919ea53d58\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"
Jul 15 19:22:47.124119 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.124096    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"]
Jul 15 19:22:47.208023 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.207996    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp4ph\" (UniqueName: \"kubernetes.io/projected/acf759f2-50d0-45f3-a138-d8919ea53d58-kube-api-access-fp4ph\") pod \"llmisvc-controller-manager-777c6bd7d5-s8vxc\" (UID: \"acf759f2-50d0-45f3-a138-d8919ea53d58\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"
Jul 15 19:22:47.208178 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.208047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acf759f2-50d0-45f3-a138-d8919ea53d58-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-s8vxc\" (UID: \"acf759f2-50d0-45f3-a138-d8919ea53d58\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"
Jul 15 19:22:47.208178 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:22:47.208160    2560 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 15 19:22:47.208282 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:22:47.208211    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acf759f2-50d0-45f3-a138-d8919ea53d58-cert podName:acf759f2-50d0-45f3-a138-d8919ea53d58 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:47.708196043 +0000 UTC m=+439.778055418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/acf759f2-50d0-45f3-a138-d8919ea53d58-cert") pod "llmisvc-controller-manager-777c6bd7d5-s8vxc" (UID: "acf759f2-50d0-45f3-a138-d8919ea53d58") : secret "llmisvc-webhook-server-cert" not found
Jul 15 19:22:47.225747 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.225675    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:47.232623 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.232600    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp4ph\" (UniqueName: \"kubernetes.io/projected/acf759f2-50d0-45f3-a138-d8919ea53d58-kube-api-access-fp4ph\") pod \"llmisvc-controller-manager-777c6bd7d5-s8vxc\" (UID: \"acf759f2-50d0-45f3-a138-d8919ea53d58\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"
Jul 15 19:22:47.711942 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.711906    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acf759f2-50d0-45f3-a138-d8919ea53d58-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-s8vxc\" (UID: \"acf759f2-50d0-45f3-a138-d8919ea53d58\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"
Jul 15 19:22:47.714247 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.714224    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acf759f2-50d0-45f3-a138-d8919ea53d58-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-s8vxc\" (UID: \"acf759f2-50d0-45f3-a138-d8919ea53d58\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"
Jul 15 19:22:47.835828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:47.835745    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:48.001346 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.001313    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"
Jul 15 19:22:48.158710 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.158679    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"]
Jul 15 19:22:48.163633 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:22:48.163603    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podacf759f2_50d0_45f3_a138_d8919ea53d58.slice/crio-038f0598ee8283c3fa87817cedbf22a45b406112ae411f3e10d03fbf792d8081 WatchSource:0}: Error finding container 038f0598ee8283c3fa87817cedbf22a45b406112ae411f3e10d03fbf792d8081: Status 404 returned error can't find the container with id 038f0598ee8283c3fa87817cedbf22a45b406112ae411f3e10d03fbf792d8081
Jul 15 19:22:48.388287 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.388208    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:48.476234 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.476198    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:48.480173 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.480145    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc" event={"ID":"acf759f2-50d0-45f3-a138-d8919ea53d58","Type":"ContainerStarted","Data":"038f0598ee8283c3fa87817cedbf22a45b406112ae411f3e10d03fbf792d8081"}
Jul 15 19:22:48.500322 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.500300    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"]
Jul 15 19:22:48.503574 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.503557    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"
Jul 15 19:22:48.507394 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.507375    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 19:22:48.507514 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.507462    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 15 19:22:48.508587 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.508570    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 15 19:22:48.508587 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.508576    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 19:22:48.508752 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.508638    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-b2htb\""
Jul 15 19:22:48.517811 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.517785    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wtqd\" (UniqueName: \"kubernetes.io/projected/136d57a8-1783-4d0d-9a15-77b54b17b3e2-kube-api-access-7wtqd\") pod \"kube-auth-proxy-59b767f7ff-8s2cv\" (UID: \"136d57a8-1783-4d0d-9a15-77b54b17b3e2\") " pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"
Jul 15 19:22:48.517923 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.517849    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/136d57a8-1783-4d0d-9a15-77b54b17b3e2-tls-certs\") pod \"kube-auth-proxy-59b767f7ff-8s2cv\" (UID: \"136d57a8-1783-4d0d-9a15-77b54b17b3e2\") " pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"
Jul 15 19:22:48.517923 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.517886    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/136d57a8-1783-4d0d-9a15-77b54b17b3e2-tmp\") pod \"kube-auth-proxy-59b767f7ff-8s2cv\" (UID: \"136d57a8-1783-4d0d-9a15-77b54b17b3e2\") " pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"
Jul 15 19:22:48.519692 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.519670    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"]
Jul 15 19:22:48.575716 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.575692    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:48.618991 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.618963    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wtqd\" (UniqueName: \"kubernetes.io/projected/136d57a8-1783-4d0d-9a15-77b54b17b3e2-kube-api-access-7wtqd\") pod \"kube-auth-proxy-59b767f7ff-8s2cv\" (UID: \"136d57a8-1783-4d0d-9a15-77b54b17b3e2\") " pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"
Jul 15 19:22:48.619083 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.619009    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/136d57a8-1783-4d0d-9a15-77b54b17b3e2-tls-certs\") pod \"kube-auth-proxy-59b767f7ff-8s2cv\" (UID: \"136d57a8-1783-4d0d-9a15-77b54b17b3e2\") " pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"
Jul 15 19:22:48.619083 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.619040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/136d57a8-1783-4d0d-9a15-77b54b17b3e2-tmp\") pod \"kube-auth-proxy-59b767f7ff-8s2cv\" (UID: \"136d57a8-1783-4d0d-9a15-77b54b17b3e2\") " pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"
Jul 15 19:22:48.619316 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:22:48.619167    2560 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 15 19:22:48.619316 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:22:48.619248    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/136d57a8-1783-4d0d-9a15-77b54b17b3e2-tls-certs podName:136d57a8-1783-4d0d-9a15-77b54b17b3e2 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:49.119233964 +0000 UTC m=+441.189093350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/136d57a8-1783-4d0d-9a15-77b54b17b3e2-tls-certs") pod "kube-auth-proxy-59b767f7ff-8s2cv" (UID: "136d57a8-1783-4d0d-9a15-77b54b17b3e2") : secret "kube-auth-proxy-tls" not found
Jul 15 19:22:48.621224 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.621205    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/136d57a8-1783-4d0d-9a15-77b54b17b3e2-tmp\") pod \"kube-auth-proxy-59b767f7ff-8s2cv\" (UID: \"136d57a8-1783-4d0d-9a15-77b54b17b3e2\") " pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"
Jul 15 19:22:48.630140 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.630115    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wtqd\" (UniqueName: \"kubernetes.io/projected/136d57a8-1783-4d0d-9a15-77b54b17b3e2-kube-api-access-7wtqd\") pod \"kube-auth-proxy-59b767f7ff-8s2cv\" (UID: \"136d57a8-1783-4d0d-9a15-77b54b17b3e2\") " pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"
Jul 15 19:22:48.970966 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:48.970933    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:49.124528 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:49.124497    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/136d57a8-1783-4d0d-9a15-77b54b17b3e2-tls-certs\") pod \"kube-auth-proxy-59b767f7ff-8s2cv\" (UID: \"136d57a8-1783-4d0d-9a15-77b54b17b3e2\") " pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"
Jul 15 19:22:49.126596 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:49.126579    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/136d57a8-1783-4d0d-9a15-77b54b17b3e2-tls-certs\") pod \"kube-auth-proxy-59b767f7ff-8s2cv\" (UID: \"136d57a8-1783-4d0d-9a15-77b54b17b3e2\") " pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"
Jul 15 19:22:49.415002 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:49.414970    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"
Jul 15 19:22:49.563029 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:49.562999    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv"]
Jul 15 19:22:49.564396 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:22:49.564371    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136d57a8_1783_4d0d_9a15_77b54b17b3e2.slice/crio-b7362f4a7bec33ec721091762d3b9c35cbd4c4881f96ff8fe66df7adac44b45e WatchSource:0}: Error finding container b7362f4a7bec33ec721091762d3b9c35cbd4c4881f96ff8fe66df7adac44b45e: Status 404 returned error can't find the container with id b7362f4a7bec33ec721091762d3b9c35cbd4c4881f96ff8fe66df7adac44b45e
Jul 15 19:22:50.318022 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:50.317918    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:50.394341 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:50.394311    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:50.496563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:50.496522    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv" event={"ID":"136d57a8-1783-4d0d-9a15-77b54b17b3e2","Type":"ContainerStarted","Data":"b7362f4a7bec33ec721091762d3b9c35cbd4c4881f96ff8fe66df7adac44b45e"}
Jul 15 19:22:51.501266 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:51.501237    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc" event={"ID":"acf759f2-50d0-45f3-a138-d8919ea53d58","Type":"ContainerStarted","Data":"464ea4aee84cc720463a9ec3ed11d9a07228b52cb8beacdde395a368decfadfc"}
Jul 15 19:22:51.501663 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:51.501338    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"
Jul 15 19:22:51.528720 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:51.525424    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc" podStartSLOduration=1.466604946 podStartE2EDuration="4.525406444s" totalImagesPullingTime="3.058801498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:22:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:51.522887733 +0000 UTC m=+443.592747130" watchObservedRunningTime="2026-07-15 19:22:51.525406444 +0000 UTC m=+443.595265841"
Jul 15 19:22:51.679774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:51.679746    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:52.506304 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:52.506262    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv" event={"ID":"136d57a8-1783-4d0d-9a15-77b54b17b3e2","Type":"ContainerStarted","Data":"9d7ec5517d43587b6e685ed8962a7454bb3462720026f2e6a0ea49ce7b86d29d"}
Jul 15 19:22:52.523987 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:52.523945    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-59b767f7ff-8s2cv" podStartSLOduration=1.806497206 podStartE2EDuration="4.523930103s" totalImagesPullingTime="2.717432897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:52.521614485 +0000 UTC m=+444.591473882" watchObservedRunningTime="2026-07-15 19:22:52.523930103 +0000 UTC m=+444.593789500"
Jul 15 19:22:54.096311 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:54.096277    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:54.195215 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:54.195167    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:54.399698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:54.399616    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:54.464632 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:54.464601    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-mm4c8"
Jul 15 19:22:54.466461 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:54.466436    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-78lfd"
Jul 15 19:22:54.498794 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:54.498758    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:22:54.635544 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:22:54.635512    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:23:08.627257 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.627223    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67cc4c5686-jcz9v"]
Jul 15 19:23:08.629868 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.629845    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.640695 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.640672    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67cc4c5686-jcz9v"]
Jul 15 19:23:08.698389 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.698356    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c680f8c-e12d-4c2f-9b93-ce802fd13841-console-serving-cert\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.698584 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.698400    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c680f8c-e12d-4c2f-9b93-ce802fd13841-console-config\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.698584 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.698475    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c680f8c-e12d-4c2f-9b93-ce802fd13841-console-oauth-config\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.698584 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.698516    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c680f8c-e12d-4c2f-9b93-ce802fd13841-service-ca\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.698695 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.698617    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c680f8c-e12d-4c2f-9b93-ce802fd13841-oauth-serving-cert\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.698695 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.698635    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6glwk\" (UniqueName: \"kubernetes.io/projected/3c680f8c-e12d-4c2f-9b93-ce802fd13841-kube-api-access-6glwk\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.698695 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.698655    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c680f8c-e12d-4c2f-9b93-ce802fd13841-trusted-ca-bundle\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.720098 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.720068    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:23:08.799584 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.799562    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c680f8c-e12d-4c2f-9b93-ce802fd13841-console-serving-cert\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.799678 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.799594    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c680f8c-e12d-4c2f-9b93-ce802fd13841-console-config\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.799678 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.799620    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c680f8c-e12d-4c2f-9b93-ce802fd13841-console-oauth-config\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.799760 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.799721    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c680f8c-e12d-4c2f-9b93-ce802fd13841-service-ca\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.799855 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.799840    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c680f8c-e12d-4c2f-9b93-ce802fd13841-oauth-serving-cert\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.799889 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.799868    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6glwk\" (UniqueName: \"kubernetes.io/projected/3c680f8c-e12d-4c2f-9b93-ce802fd13841-kube-api-access-6glwk\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.799929 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.799902    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c680f8c-e12d-4c2f-9b93-ce802fd13841-trusted-ca-bundle\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.800385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.800361    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c680f8c-e12d-4c2f-9b93-ce802fd13841-console-config\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.800546 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.800523    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c680f8c-e12d-4c2f-9b93-ce802fd13841-oauth-serving-cert\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.800546 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.800415    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c680f8c-e12d-4c2f-9b93-ce802fd13841-service-ca\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.800737 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.800717    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c680f8c-e12d-4c2f-9b93-ce802fd13841-trusted-ca-bundle\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.802001 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.801980    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c680f8c-e12d-4c2f-9b93-ce802fd13841-console-oauth-config\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.802139 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.802119    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c680f8c-e12d-4c2f-9b93-ce802fd13841-console-serving-cert\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.808618 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.808599    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6glwk\" (UniqueName: \"kubernetes.io/projected/3c680f8c-e12d-4c2f-9b93-ce802fd13841-kube-api-access-6glwk\") pod \"console-67cc4c5686-jcz9v\" (UID: \"3c680f8c-e12d-4c2f-9b93-ce802fd13841\") " pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:08.821856 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.821836    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:23:08.921881 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.921795    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:23:08.939286 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:08.939259    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:09.065217 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:09.065183    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67cc4c5686-jcz9v"]
Jul 15 19:23:09.068525 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:23:09.068498    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c680f8c_e12d_4c2f_9b93_ce802fd13841.slice/crio-3963fe6c467b3c3f56d1ed904f898cd341f156ac9673eab7b6644e80549a04fa WatchSource:0}: Error finding container 3963fe6c467b3c3f56d1ed904f898cd341f156ac9673eab7b6644e80549a04fa: Status 404 returned error can't find the container with id 3963fe6c467b3c3f56d1ed904f898cd341f156ac9673eab7b6644e80549a04fa
Jul 15 19:23:09.421027 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:09.420992    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:23:09.521785 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:09.521749    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:23:09.583146 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:09.583117    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67cc4c5686-jcz9v" event={"ID":"3c680f8c-e12d-4c2f-9b93-ce802fd13841","Type":"ContainerStarted","Data":"90379578a5d4ff3ee93b363a3aa444b904d8328f2472788baea502db8f0a7218"}
Jul 15 19:23:09.583304 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:09.583168    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67cc4c5686-jcz9v" event={"ID":"3c680f8c-e12d-4c2f-9b93-ce802fd13841","Type":"ContainerStarted","Data":"3963fe6c467b3c3f56d1ed904f898cd341f156ac9673eab7b6644e80549a04fa"}
Jul 15 19:23:09.603245 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:09.603199    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67cc4c5686-jcz9v" podStartSLOduration=1.603184532 podStartE2EDuration="1.603184532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:23:09.601669273 +0000 UTC m=+461.671528669" watchObservedRunningTime="2026-07-15 19:23:09.603184532 +0000 UTC m=+461.673043928"
Jul 15 19:23:18.939417 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:18.939381    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:18.939808 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:18.939435    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:18.944067 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:18.944046    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:19.635491 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:19.635463    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67cc4c5686-jcz9v"
Jul 15 19:23:19.680391 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:19.680353    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d9c556586-6v2lz"]
Jul 15 19:23:19.826970 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:19.826936    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:23:19.944651 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:19.944539    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:23:22.508571 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:22.508542    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s8vxc"
Jul 15 19:23:38.920590 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:38.920543    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:23:39.026034 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:39.025983    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:23:39.121764 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:39.121722    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:23:44.700623 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:44.700562    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d9c556586-6v2lz" podUID="e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f" containerName="console" containerID="cri-o://9fb0e2ca33c1c343435089e4b5481564fe7bbd986a697331db55d876b9b5b24c" gracePeriod=15
Jul 15 19:23:44.934814 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:44.934785    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:23:45.005773 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.005747    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-config\") pod \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") "
Jul 15 19:23:45.005928 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.005811    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-oauth-config\") pod \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") "
Jul 15 19:23:45.005928 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.005885    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-service-ca\") pod \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") "
Jul 15 19:23:45.005928 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.005923    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-serving-cert\") pod \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") "
Jul 15 19:23:45.006096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.005954    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-oauth-serving-cert\") pod \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") "
Jul 15 19:23:45.006096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.005984    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87w98\" (UniqueName: \"kubernetes.io/projected/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-kube-api-access-87w98\") pod \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") "
Jul 15 19:23:45.006096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.006010    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-trusted-ca-bundle\") pod \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\" (UID: \"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f\") "
Jul 15 19:23:45.006231 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.006151    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-config" (OuterVolumeSpecName: "console-config") pod "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f" (UID: "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:23:45.006289 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.006237    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-service-ca" (OuterVolumeSpecName: "service-ca") pod "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f" (UID: "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:23:45.006345 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.006305    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:23:45.006345 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.006322    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-service-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:23:45.006584 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.006563    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f" (UID: "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:23:45.006584 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.006567    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f" (UID: "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:23:45.008356 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.008331    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f" (UID: "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:23:45.008591 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.008538    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f" (UID: "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:23:45.008679 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.008593    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-kube-api-access-87w98" (OuterVolumeSpecName: "kube-api-access-87w98") pod "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f" (UID: "e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f"). InnerVolumeSpecName "kube-api-access-87w98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:23:45.106802 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.106780    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-oauth-config\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:23:45.106802 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.106800    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-console-serving-cert\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:23:45.106933 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.106810    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-oauth-serving-cert\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:23:45.106933 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.106818    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87w98\" (UniqueName: \"kubernetes.io/projected/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-kube-api-access-87w98\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:23:45.106933 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.106835    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f-trusted-ca-bundle\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:23:45.744808 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.744780    2560 generic.go:356] "Generic (PLEG): container finished" podID="e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f" containerID="9fb0e2ca33c1c343435089e4b5481564fe7bbd986a697331db55d876b9b5b24c" exitCode=0
Jul 15 19:23:45.745234 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.744844    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d9c556586-6v2lz"
Jul 15 19:23:45.745234 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.744855    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d9c556586-6v2lz" event={"ID":"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f","Type":"ContainerDied","Data":"9fb0e2ca33c1c343435089e4b5481564fe7bbd986a697331db55d876b9b5b24c"}
Jul 15 19:23:45.745234 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.744903    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d9c556586-6v2lz" event={"ID":"e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f","Type":"ContainerDied","Data":"d352aff755396a0e88ccb5461ff39eadbb49ac90553117c41b066bd8b2ce3ba1"}
Jul 15 19:23:45.745234 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.744922    2560 scope.go:117] "RemoveContainer" containerID="9fb0e2ca33c1c343435089e4b5481564fe7bbd986a697331db55d876b9b5b24c"
Jul 15 19:23:45.753232 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.753219    2560 scope.go:117] "RemoveContainer" containerID="9fb0e2ca33c1c343435089e4b5481564fe7bbd986a697331db55d876b9b5b24c"
Jul 15 19:23:45.753545 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:23:45.753528    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fb0e2ca33c1c343435089e4b5481564fe7bbd986a697331db55d876b9b5b24c\": container with ID starting with 9fb0e2ca33c1c343435089e4b5481564fe7bbd986a697331db55d876b9b5b24c not found: ID does not exist" containerID="9fb0e2ca33c1c343435089e4b5481564fe7bbd986a697331db55d876b9b5b24c"
Jul 15 19:23:45.753598 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.753553    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fb0e2ca33c1c343435089e4b5481564fe7bbd986a697331db55d876b9b5b24c"} err="failed to get container status \"9fb0e2ca33c1c343435089e4b5481564fe7bbd986a697331db55d876b9b5b24c\": rpc error: code = NotFound desc = could not find container \"9fb0e2ca33c1c343435089e4b5481564fe7bbd986a697331db55d876b9b5b24c\": container with ID starting with 9fb0e2ca33c1c343435089e4b5481564fe7bbd986a697331db55d876b9b5b24c not found: ID does not exist"
Jul 15 19:23:45.764881 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.764857    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d9c556586-6v2lz"]
Jul 15 19:23:45.770842 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:45.770812    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d9c556586-6v2lz"]
Jul 15 19:23:46.494988 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:46.494958    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f" path="/var/lib/kubelet/pods/e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f/volumes"
Jul 15 19:23:56.021745 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:23:56.021714    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:05.396140 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:05.396104    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:12.425072 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:12.425002    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:15.411058 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:15.411024    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:16.258328 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:16.258295    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:17.811413 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:17.811378    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:18.003824 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:18.003794    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:18.610636 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:18.610604    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:19.455770 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:19.455734    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:20.201303 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:20.201233    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:20.445619 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:20.445584    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:22.038656 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:22.038628    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:23.862605 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:23.862567    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:23.966629 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:23.966586    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:29.192961 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:29.192926    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:29.304658 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:29.304626    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:30.516611 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.516575    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"]
Jul 15 19:24:30.517082 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.517063    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f" containerName="console"
Jul 15 19:24:30.517156 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.517085    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f" containerName="console"
Jul 15 19:24:30.517212 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.517186    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9dee4f2-cfa8-4c0e-91b2-b0eeff3b912f" containerName="console"
Jul 15 19:24:30.519917 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.519896    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"
Jul 15 19:24:30.523284 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.523259    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 19:24:30.523385 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.523369    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 19:24:30.523473 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.523394    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-288zc\""
Jul 15 19:24:30.536369 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.536348    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"]
Jul 15 19:24:30.570943 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.570913    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"]
Jul 15 19:24:30.571169 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:24:30.571150    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-2kjcm], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt" podUID="789892f4-73c5-43b2-af5b-c0c6c129270e"
Jul 15 19:24:30.576001 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.575976    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"]
Jul 15 19:24:30.592187 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.592161    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/789892f4-73c5-43b2-af5b-c0c6c129270e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j2crt\" (UID: \"789892f4-73c5-43b2-af5b-c0c6c129270e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"
Jul 15 19:24:30.592322 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.592222    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kjcm\" (UniqueName: \"kubernetes.io/projected/789892f4-73c5-43b2-af5b-c0c6c129270e-kube-api-access-2kjcm\") pod \"kuadrant-operator-controller-manager-86878c769f-j2crt\" (UID: \"789892f4-73c5-43b2-af5b-c0c6c129270e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"
Jul 15 19:24:30.603934 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.603909    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"]
Jul 15 19:24:30.606542 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.606523    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"
Jul 15 19:24:30.621550 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.621524    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"]
Jul 15 19:24:30.693585 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.693551    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/789892f4-73c5-43b2-af5b-c0c6c129270e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j2crt\" (UID: \"789892f4-73c5-43b2-af5b-c0c6c129270e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"
Jul 15 19:24:30.693585 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.693586    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd753323-43e7-4d08-b740-00e3386448d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xqwzk\" (UID: \"dd753323-43e7-4d08-b740-00e3386448d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"
Jul 15 19:24:30.693822 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.693625    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kjcm\" (UniqueName: \"kubernetes.io/projected/789892f4-73c5-43b2-af5b-c0c6c129270e-kube-api-access-2kjcm\") pod \"kuadrant-operator-controller-manager-86878c769f-j2crt\" (UID: \"789892f4-73c5-43b2-af5b-c0c6c129270e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"
Jul 15 19:24:30.693822 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.693722    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ljl5\" (UniqueName: \"kubernetes.io/projected/dd753323-43e7-4d08-b740-00e3386448d8-kube-api-access-9ljl5\") pod \"kuadrant-operator-controller-manager-86878c769f-xqwzk\" (UID: \"dd753323-43e7-4d08-b740-00e3386448d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"
Jul 15 19:24:30.693999 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.693978    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/789892f4-73c5-43b2-af5b-c0c6c129270e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j2crt\" (UID: \"789892f4-73c5-43b2-af5b-c0c6c129270e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"
Jul 15 19:24:30.700226 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:24:30.700204    2560 projected.go:196] Error preparing data for projected volume kube-api-access-2kjcm for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-j2crt" not found
Jul 15 19:24:30.700309 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:24:30.700264    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/789892f4-73c5-43b2-af5b-c0c6c129270e-kube-api-access-2kjcm podName:789892f4-73c5-43b2-af5b-c0c6c129270e nodeName:}" failed. No retries permitted until 2026-07-15 19:24:31.200244318 +0000 UTC m=+543.270103701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2kjcm" (UniqueName: "kubernetes.io/projected/789892f4-73c5-43b2-af5b-c0c6c129270e-kube-api-access-2kjcm") pod "kuadrant-operator-controller-manager-86878c769f-j2crt" (UID: "789892f4-73c5-43b2-af5b-c0c6c129270e") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-j2crt" not found
Jul 15 19:24:30.726734 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.726707    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:30.794686 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.794610    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ljl5\" (UniqueName: \"kubernetes.io/projected/dd753323-43e7-4d08-b740-00e3386448d8-kube-api-access-9ljl5\") pod \"kuadrant-operator-controller-manager-86878c769f-xqwzk\" (UID: \"dd753323-43e7-4d08-b740-00e3386448d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"
Jul 15 19:24:30.794686 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.794664    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd753323-43e7-4d08-b740-00e3386448d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xqwzk\" (UID: \"dd753323-43e7-4d08-b740-00e3386448d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"
Jul 15 19:24:30.795055 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.795038    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd753323-43e7-4d08-b740-00e3386448d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xqwzk\" (UID: \"dd753323-43e7-4d08-b740-00e3386448d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"
Jul 15 19:24:30.813126 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.813103    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ljl5\" (UniqueName: \"kubernetes.io/projected/dd753323-43e7-4d08-b740-00e3386448d8-kube-api-access-9ljl5\") pod \"kuadrant-operator-controller-manager-86878c769f-xqwzk\" (UID: \"dd753323-43e7-4d08-b740-00e3386448d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"
Jul 15 19:24:30.834221 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.834196    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:30.917830 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.917799    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"
Jul 15 19:24:30.946869 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.946841    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"
Jul 15 19:24:30.949331 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:24:30.949297    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-j2crt\" is forbidden: User \"system:node:ip-10-0-143-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-114.ec2.internal' and this object" podUID="789892f4-73c5-43b2-af5b-c0c6c129270e" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"
Jul 15 19:24:30.953288 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.953266    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"
Jul 15 19:24:30.955384 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:24:30.955356    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-j2crt\" is forbidden: User \"system:node:ip-10-0-143-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-114.ec2.internal' and this object" podUID="789892f4-73c5-43b2-af5b-c0c6c129270e" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"
Jul 15 19:24:30.996709 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.996618    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/789892f4-73c5-43b2-af5b-c0c6c129270e-extensions-socket-volume\") pod \"789892f4-73c5-43b2-af5b-c0c6c129270e\" (UID: \"789892f4-73c5-43b2-af5b-c0c6c129270e\") "
Jul 15 19:24:30.996885 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.996862    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/789892f4-73c5-43b2-af5b-c0c6c129270e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "789892f4-73c5-43b2-af5b-c0c6c129270e" (UID: "789892f4-73c5-43b2-af5b-c0c6c129270e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:24:30.996950 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:30.996869    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kjcm\" (UniqueName: \"kubernetes.io/projected/789892f4-73c5-43b2-af5b-c0c6c129270e-kube-api-access-2kjcm\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:24:31.048734 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:31.048711    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"]
Jul 15 19:24:31.051393 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:24:31.051369    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd753323_43e7_4d08_b740_00e3386448d8.slice/crio-c314fbefa81d10b3e90da7462cb90d452a7d1fc5abe5545402f50be6349a557b WatchSource:0}: Error finding container c314fbefa81d10b3e90da7462cb90d452a7d1fc5abe5545402f50be6349a557b: Status 404 returned error can't find the container with id c314fbefa81d10b3e90da7462cb90d452a7d1fc5abe5545402f50be6349a557b
Jul 15 19:24:31.098055 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:31.098028    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/789892f4-73c5-43b2-af5b-c0c6c129270e-extensions-socket-volume\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:24:31.955419 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:31.955369    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk" event={"ID":"dd753323-43e7-4d08-b740-00e3386448d8","Type":"ContainerStarted","Data":"c314fbefa81d10b3e90da7462cb90d452a7d1fc5abe5545402f50be6349a557b"}
Jul 15 19:24:31.955944 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:31.955391    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"
Jul 15 19:24:31.957998 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:24:31.957957    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-j2crt\" is forbidden: User \"system:node:ip-10-0-143-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-114.ec2.internal' and this object" podUID="789892f4-73c5-43b2-af5b-c0c6c129270e" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"
Jul 15 19:24:31.965080 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:24:31.965047    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-j2crt\" is forbidden: User \"system:node:ip-10-0-143-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-114.ec2.internal' and this object" podUID="789892f4-73c5-43b2-af5b-c0c6c129270e" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2crt"
Jul 15 19:24:32.496532 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:32.496503    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="789892f4-73c5-43b2-af5b-c0c6c129270e" path="/var/lib/kubelet/pods/789892f4-73c5-43b2-af5b-c0c6c129270e/volumes"
Jul 15 19:24:35.426727 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:35.426686    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:35.529268 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:35.529237    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:35.977673 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:35.977565    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk" event={"ID":"dd753323-43e7-4d08-b740-00e3386448d8","Type":"ContainerStarted","Data":"d2323c743b3f748a152ebd17a3fe0a0fe42127c06fd8f08b1dca67b21a14f653"}
Jul 15 19:24:35.977673 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:35.977658    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"
Jul 15 19:24:35.997947 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:35.997892    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk" podStartSLOduration=1.400369492 podStartE2EDuration="5.997875262s" totalImagesPullingTime="4.59750577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:24:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:24:35.996246704 +0000 UTC m=+548.066106101" watchObservedRunningTime="2026-07-15 19:24:35.997875262 +0000 UTC m=+548.067734660"
Jul 15 19:24:40.221330 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:40.221296    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:40.322047 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:40.322014    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:44.023485 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:44.023437    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:44.123919 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:44.123880    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:46.984617 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:46.984586    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"
Jul 15 19:24:50.296513 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:50.296482    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:50.395313 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:50.395279    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:55.049092 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:55.049051    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:55.150922 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:55.150892    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:59.850819 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:59.850779    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:24:59.950774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:24:59.950735    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:03.520026 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:03.519995    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:03.622680 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:03.622647    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:04.056172 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:04.056132    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:04.340444 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:04.340366    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:05.093848 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:05.093818    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:05.200577 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:05.200538    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:05.258329 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:05.258296    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:05.875275 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:05.875241    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-mplxn"]
Jul 15 19:25:05.877947 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:05.877926    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mplxn"
Jul 15 19:25:05.880383 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:05.880364    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-r9c97\""
Jul 15 19:25:05.885590 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:05.885561    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mplxn"]
Jul 15 19:25:05.911487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:05.911438    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t2sd\" (UniqueName: \"kubernetes.io/projected/f06c54b7-8682-4ba2-8796-6ba477d6f48c-kube-api-access-8t2sd\") pod \"authorino-58cd955675-mplxn\" (UID: \"f06c54b7-8682-4ba2-8796-6ba477d6f48c\") " pod="kuadrant-system/authorino-58cd955675-mplxn"
Jul 15 19:25:06.012982 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:06.012948    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t2sd\" (UniqueName: \"kubernetes.io/projected/f06c54b7-8682-4ba2-8796-6ba477d6f48c-kube-api-access-8t2sd\") pod \"authorino-58cd955675-mplxn\" (UID: \"f06c54b7-8682-4ba2-8796-6ba477d6f48c\") " pod="kuadrant-system/authorino-58cd955675-mplxn"
Jul 15 19:25:06.021374 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:06.021345    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t2sd\" (UniqueName: \"kubernetes.io/projected/f06c54b7-8682-4ba2-8796-6ba477d6f48c-kube-api-access-8t2sd\") pod \"authorino-58cd955675-mplxn\" (UID: \"f06c54b7-8682-4ba2-8796-6ba477d6f48c\") " pod="kuadrant-system/authorino-58cd955675-mplxn"
Jul 15 19:25:06.205131 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:06.205040    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mplxn"
Jul 15 19:25:06.327504 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:06.327480    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mplxn"]
Jul 15 19:25:06.329359 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:25:06.329332    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf06c54b7_8682_4ba2_8796_6ba477d6f48c.slice/crio-ddeca059f9122e05fd2a685695d671b27f7ca9e0d6b9e6f3015528311d444302 WatchSource:0}: Error finding container ddeca059f9122e05fd2a685695d671b27f7ca9e0d6b9e6f3015528311d444302: Status 404 returned error can't find the container with id ddeca059f9122e05fd2a685695d671b27f7ca9e0d6b9e6f3015528311d444302
Jul 15 19:25:06.393500 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:06.393465    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:06.494013 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:06.493971    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:07.113246 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:07.113211    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mplxn" event={"ID":"f06c54b7-8682-4ba2-8796-6ba477d6f48c","Type":"ContainerStarted","Data":"ddeca059f9122e05fd2a685695d671b27f7ca9e0d6b9e6f3015528311d444302"}
Jul 15 19:25:10.130125 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:10.130085    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mplxn" event={"ID":"f06c54b7-8682-4ba2-8796-6ba477d6f48c","Type":"ContainerStarted","Data":"e981db49aed28ece4fa8503d5c5ebea717a9671d3912aa92530c3cc4f801d025"}
Jul 15 19:25:10.145141 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:10.145089    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-mplxn" podStartSLOduration=1.916493161 podStartE2EDuration="5.145074768s" totalImagesPullingTime="3.228581607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:10.144360339 +0000 UTC m=+582.214219735" watchObservedRunningTime="2026-07-15 19:25:10.145074768 +0000 UTC m=+582.214934164"
Jul 15 19:25:11.250972 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:11.250923    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:11.350825 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:11.350768    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:13.098619 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:13.098582    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mplxn"]
Jul 15 19:25:13.099018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:13.098753    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-mplxn" podUID="f06c54b7-8682-4ba2-8796-6ba477d6f48c" containerName="authorino" containerID="cri-o://e981db49aed28ece4fa8503d5c5ebea717a9671d3912aa92530c3cc4f801d025" gracePeriod=30
Jul 15 19:25:13.341139 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:13.341114    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mplxn"
Jul 15 19:25:13.380600 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:13.380525    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t2sd\" (UniqueName: \"kubernetes.io/projected/f06c54b7-8682-4ba2-8796-6ba477d6f48c-kube-api-access-8t2sd\") pod \"f06c54b7-8682-4ba2-8796-6ba477d6f48c\" (UID: \"f06c54b7-8682-4ba2-8796-6ba477d6f48c\") "
Jul 15 19:25:13.382679 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:13.382647    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f06c54b7-8682-4ba2-8796-6ba477d6f48c-kube-api-access-8t2sd" (OuterVolumeSpecName: "kube-api-access-8t2sd") pod "f06c54b7-8682-4ba2-8796-6ba477d6f48c" (UID: "f06c54b7-8682-4ba2-8796-6ba477d6f48c"). InnerVolumeSpecName "kube-api-access-8t2sd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:25:13.481656 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:13.481617    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t2sd\" (UniqueName: \"kubernetes.io/projected/f06c54b7-8682-4ba2-8796-6ba477d6f48c-kube-api-access-8t2sd\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:25:14.152547 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:14.152513    2560 generic.go:356] "Generic (PLEG): container finished" podID="f06c54b7-8682-4ba2-8796-6ba477d6f48c" containerID="e981db49aed28ece4fa8503d5c5ebea717a9671d3912aa92530c3cc4f801d025" exitCode=0
Jul 15 19:25:14.152920 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:14.152565    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mplxn"
Jul 15 19:25:14.152920 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:14.152568    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mplxn" event={"ID":"f06c54b7-8682-4ba2-8796-6ba477d6f48c","Type":"ContainerDied","Data":"e981db49aed28ece4fa8503d5c5ebea717a9671d3912aa92530c3cc4f801d025"}
Jul 15 19:25:14.152920 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:14.152693    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mplxn" event={"ID":"f06c54b7-8682-4ba2-8796-6ba477d6f48c","Type":"ContainerDied","Data":"ddeca059f9122e05fd2a685695d671b27f7ca9e0d6b9e6f3015528311d444302"}
Jul 15 19:25:14.152920 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:14.152724    2560 scope.go:117] "RemoveContainer" containerID="e981db49aed28ece4fa8503d5c5ebea717a9671d3912aa92530c3cc4f801d025"
Jul 15 19:25:14.161278 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:14.161259    2560 scope.go:117] "RemoveContainer" containerID="e981db49aed28ece4fa8503d5c5ebea717a9671d3912aa92530c3cc4f801d025"
Jul 15 19:25:14.161529 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:25:14.161513    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e981db49aed28ece4fa8503d5c5ebea717a9671d3912aa92530c3cc4f801d025\": container with ID starting with e981db49aed28ece4fa8503d5c5ebea717a9671d3912aa92530c3cc4f801d025 not found: ID does not exist" containerID="e981db49aed28ece4fa8503d5c5ebea717a9671d3912aa92530c3cc4f801d025"
Jul 15 19:25:14.161583 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:14.161539    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e981db49aed28ece4fa8503d5c5ebea717a9671d3912aa92530c3cc4f801d025"} err="failed to get container status \"e981db49aed28ece4fa8503d5c5ebea717a9671d3912aa92530c3cc4f801d025\": rpc error: code = NotFound desc = could not find container \"e981db49aed28ece4fa8503d5c5ebea717a9671d3912aa92530c3cc4f801d025\": container with ID starting with e981db49aed28ece4fa8503d5c5ebea717a9671d3912aa92530c3cc4f801d025 not found: ID does not exist"
Jul 15 19:25:14.173378 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:14.173356    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mplxn"]
Jul 15 19:25:14.177752 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:14.177732    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-mplxn"]
Jul 15 19:25:14.495238 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:14.495205    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f06c54b7-8682-4ba2-8796-6ba477d6f48c" path="/var/lib/kubelet/pods/f06c54b7-8682-4ba2-8796-6ba477d6f48c/volumes"
Jul 15 19:25:15.266340 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:15.266305    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:15.369404 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:15.369373    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:20.078956 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:20.078910    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:20.177867 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:20.177840    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:22.638186 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:22.638144    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:24.544812 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:24.544775    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:24.646118 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:24.646086    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:26.326357 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:26.326320    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:26.424350 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:26.424320    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:26.799031 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:26.798994    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:28.376809 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:28.376786    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:25:28.386483 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:28.386462    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:25:36.228668 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:36.228630    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:36.330350 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:36.330319    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:39.536699 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:39.536666    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:40.207841 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:40.207807    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:41.483375 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.483344    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6796f97bff-8m6p8"]
Jul 15 19:25:41.483803 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.483699    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f06c54b7-8682-4ba2-8796-6ba477d6f48c" containerName="authorino"
Jul 15 19:25:41.483803 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.483712    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f06c54b7-8682-4ba2-8796-6ba477d6f48c" containerName="authorino"
Jul 15 19:25:41.483874 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.483809    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f06c54b7-8682-4ba2-8796-6ba477d6f48c" containerName="authorino"
Jul 15 19:25:41.487094 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.487074    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6796f97bff-8m6p8"
Jul 15 19:25:41.489493 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.489472    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 15 19:25:41.489601 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.489474    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-r9c97\""
Jul 15 19:25:41.493767 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.493742    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6796f97bff-8m6p8"]
Jul 15 19:25:41.533430 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.533405    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5b89a1f6-cfbf-4409-bcdd-8e661bc88be5-tls-cert\") pod \"authorino-6796f97bff-8m6p8\" (UID: \"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5\") " pod="kuadrant-system/authorino-6796f97bff-8m6p8"
Jul 15 19:25:41.533582 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.533472    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx4cl\" (UniqueName: \"kubernetes.io/projected/5b89a1f6-cfbf-4409-bcdd-8e661bc88be5-kube-api-access-hx4cl\") pod \"authorino-6796f97bff-8m6p8\" (UID: \"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5\") " pod="kuadrant-system/authorino-6796f97bff-8m6p8"
Jul 15 19:25:41.634491 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.634445    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx4cl\" (UniqueName: \"kubernetes.io/projected/5b89a1f6-cfbf-4409-bcdd-8e661bc88be5-kube-api-access-hx4cl\") pod \"authorino-6796f97bff-8m6p8\" (UID: \"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5\") " pod="kuadrant-system/authorino-6796f97bff-8m6p8"
Jul 15 19:25:41.634652 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.634539    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5b89a1f6-cfbf-4409-bcdd-8e661bc88be5-tls-cert\") pod \"authorino-6796f97bff-8m6p8\" (UID: \"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5\") " pod="kuadrant-system/authorino-6796f97bff-8m6p8"
Jul 15 19:25:41.636747 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.636728    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5b89a1f6-cfbf-4409-bcdd-8e661bc88be5-tls-cert\") pod \"authorino-6796f97bff-8m6p8\" (UID: \"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5\") " pod="kuadrant-system/authorino-6796f97bff-8m6p8"
Jul 15 19:25:41.642616 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.642594    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx4cl\" (UniqueName: \"kubernetes.io/projected/5b89a1f6-cfbf-4409-bcdd-8e661bc88be5-kube-api-access-hx4cl\") pod \"authorino-6796f97bff-8m6p8\" (UID: \"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5\") " pod="kuadrant-system/authorino-6796f97bff-8m6p8"
Jul 15 19:25:41.808446 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.808367    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6796f97bff-8m6p8"
Jul 15 19:25:41.930074 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:41.930050    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6796f97bff-8m6p8"]
Jul 15 19:25:41.932827 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:25:41.932802    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b89a1f6_cfbf_4409_bcdd_8e661bc88be5.slice/crio-6da1b22a996c7ddbf6710c94c097375ca3144faf87db3c7fd08a3e469b6a7b88 WatchSource:0}: Error finding container 6da1b22a996c7ddbf6710c94c097375ca3144faf87db3c7fd08a3e469b6a7b88: Status 404 returned error can't find the container with id 6da1b22a996c7ddbf6710c94c097375ca3144faf87db3c7fd08a3e469b6a7b88
Jul 15 19:25:42.283929 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:42.283891    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6796f97bff-8m6p8" event={"ID":"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5","Type":"ContainerStarted","Data":"6da1b22a996c7ddbf6710c94c097375ca3144faf87db3c7fd08a3e469b6a7b88"}
Jul 15 19:25:43.291135 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:43.291100    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6796f97bff-8m6p8" event={"ID":"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5","Type":"ContainerStarted","Data":"6a322232474f491ffe79bee0ce727f6b11e1dbd75442724fcb6d2c870842ac26"}
Jul 15 19:25:43.305438 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:43.305378    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6796f97bff-8m6p8" podStartSLOduration=1.965759336 podStartE2EDuration="2.305364616s" totalImagesPullingTime="339.60528ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:43.305181881 +0000 UTC m=+615.375041280" watchObservedRunningTime="2026-07-15 19:25:43.305364616 +0000 UTC m=+615.375224012"
Jul 15 19:25:46.171314 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:46.171285    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:46.269107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:46.269074    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:46.333852 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:46.333819    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:49.623126 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:49.623092    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:25:49.741311 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:25:49.741279    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:26:21.465020 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:21.464921    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:26:21.556947 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:21.556915    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:26:21.992085 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:21.992051    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:26:22.229784 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:22.229750    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:26:22.587203 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:22.587149    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:26:22.676828 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:22.676794    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:26:33.173803 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:33.173764    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:26:33.236780 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:33.236744    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:26:39.181781 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:39.181747    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:26:40.948520 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:40.948484    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"]
Jul 15 19:26:40.951613 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:40.951596    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:40.954564 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:40.954376    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 19:26:40.954564 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:40.954428    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dx2cn\""
Jul 15 19:26:40.954564 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:40.954480    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 19:26:40.955326 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:40.955309    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 15 19:26:40.961141 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:40.961120    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"]
Jul 15 19:26:40.976381 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:40.976333    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:26:41.049288 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.049247    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hff2w\" (UniqueName: \"kubernetes.io/projected/c35ec128-1999-4a20-95f2-f0961bd55d11-kube-api-access-hff2w\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.049288 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.049289    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c35ec128-1999-4a20-95f2-f0961bd55d11-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.049518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.049356    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c35ec128-1999-4a20-95f2-f0961bd55d11-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.049518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.049444    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c35ec128-1999-4a20-95f2-f0961bd55d11-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.049518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.049500    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c35ec128-1999-4a20-95f2-f0961bd55d11-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.049518 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.049516    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c35ec128-1999-4a20-95f2-f0961bd55d11-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.150436 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.150398    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c35ec128-1999-4a20-95f2-f0961bd55d11-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.150598 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.150529    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c35ec128-1999-4a20-95f2-f0961bd55d11-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.150598 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.150560    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c35ec128-1999-4a20-95f2-f0961bd55d11-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.150696 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.150630    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hff2w\" (UniqueName: \"kubernetes.io/projected/c35ec128-1999-4a20-95f2-f0961bd55d11-kube-api-access-hff2w\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.150696 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.150660    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c35ec128-1999-4a20-95f2-f0961bd55d11-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.150696 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.150684    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c35ec128-1999-4a20-95f2-f0961bd55d11-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.150950 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.150927    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c35ec128-1999-4a20-95f2-f0961bd55d11-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.151071 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.150986    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c35ec128-1999-4a20-95f2-f0961bd55d11-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.151126 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.151109    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c35ec128-1999-4a20-95f2-f0961bd55d11-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.152669 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.152647    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c35ec128-1999-4a20-95f2-f0961bd55d11-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.152972 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.152956    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c35ec128-1999-4a20-95f2-f0961bd55d11-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.158664 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.158646    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hff2w\" (UniqueName: \"kubernetes.io/projected/c35ec128-1999-4a20-95f2-f0961bd55d11-kube-api-access-hff2w\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw\" (UID: \"c35ec128-1999-4a20-95f2-f0961bd55d11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.263408 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.263387    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:41.390076 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.390045    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"]
Jul 15 19:26:41.392065 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:26:41.392039    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc35ec128_1999_4a20_95f2_f0961bd55d11.slice/crio-aaafbb1c1dc2c246636c7991714e56d30eec34eefb764f5830a0f9c4ca7fcb07 WatchSource:0}: Error finding container aaafbb1c1dc2c246636c7991714e56d30eec34eefb764f5830a0f9c4ca7fcb07: Status 404 returned error can't find the container with id aaafbb1c1dc2c246636c7991714e56d30eec34eefb764f5830a0f9c4ca7fcb07
Jul 15 19:26:41.547743 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:41.547658    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw" event={"ID":"c35ec128-1999-4a20-95f2-f0961bd55d11","Type":"ContainerStarted","Data":"aaafbb1c1dc2c246636c7991714e56d30eec34eefb764f5830a0f9c4ca7fcb07"}
Jul 15 19:26:43.560634 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:43.560602    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw" event={"ID":"c35ec128-1999-4a20-95f2-f0961bd55d11","Type":"ContainerStarted","Data":"55e33d53eb74fc891b924c59c52aefecb288a39afe6fba0e678232d6ecb1ca0d"}
Jul 15 19:26:43.578679 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:43.578631    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw" podStartSLOduration=2.319688943 podStartE2EDuration="3.57861579s" totalImagesPullingTime="1.258926847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:43.576312419 +0000 UTC m=+675.646171829" watchObservedRunningTime="2026-07-15 19:26:43.57861579 +0000 UTC m=+675.648475187"
Jul 15 19:26:48.869601 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:48.869562    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:26:51.264026 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:51.263969    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:51.264418 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:51.264049    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:51.276248 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:51.276227    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:26:51.374089 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:51.374061    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:26:51.608563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:26:51.608491    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw"
Jul 15 19:27:03.134691 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.134659    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"]
Jul 15 19:27:03.139282 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.139236    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.141756 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.141734    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 15 19:27:03.153832 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.153807    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"]
Jul 15 19:27:03.159254 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.159231    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:27:03.260425 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.260401    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89a2ec48-4d06-45f9-8cbd-a30144708001-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.260612 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.260492    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89a2ec48-4d06-45f9-8cbd-a30144708001-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.260612 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.260549    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89a2ec48-4d06-45f9-8cbd-a30144708001-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.260612 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.260588    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89a2ec48-4d06-45f9-8cbd-a30144708001-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.260612 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.260608    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89a2ec48-4d06-45f9-8cbd-a30144708001-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.260756 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.260690    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ljl9\" (UniqueName: \"kubernetes.io/projected/89a2ec48-4d06-45f9-8cbd-a30144708001-kube-api-access-2ljl9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.361749 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.361711    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89a2ec48-4d06-45f9-8cbd-a30144708001-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.361910 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.361773    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89a2ec48-4d06-45f9-8cbd-a30144708001-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.361910 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.361798    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89a2ec48-4d06-45f9-8cbd-a30144708001-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.361910 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.361814    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89a2ec48-4d06-45f9-8cbd-a30144708001-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.361910 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.361841    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ljl9\" (UniqueName: \"kubernetes.io/projected/89a2ec48-4d06-45f9-8cbd-a30144708001-kube-api-access-2ljl9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.361910 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.361866    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89a2ec48-4d06-45f9-8cbd-a30144708001-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.362183 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.362159    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89a2ec48-4d06-45f9-8cbd-a30144708001-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.362248 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.362169    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89a2ec48-4d06-45f9-8cbd-a30144708001-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.362248 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.362228    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89a2ec48-4d06-45f9-8cbd-a30144708001-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.364030 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.364006    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89a2ec48-4d06-45f9-8cbd-a30144708001-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.364258 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.364243    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89a2ec48-4d06-45f9-8cbd-a30144708001-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.369565 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.369541    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ljl9\" (UniqueName: \"kubernetes.io/projected/89a2ec48-4d06-45f9-8cbd-a30144708001-kube-api-access-2ljl9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nx8p\" (UID: \"89a2ec48-4d06-45f9-8cbd-a30144708001\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.449751 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.449671    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:03.582820 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.582793    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"]
Jul 15 19:27:03.584526 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:27:03.584497    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89a2ec48_4d06_45f9_8cbd_a30144708001.slice/crio-793bfd5dedbfaa8854595230090113081c176d62a52273c15b8fb5c04b10ae14 WatchSource:0}: Error finding container 793bfd5dedbfaa8854595230090113081c176d62a52273c15b8fb5c04b10ae14: Status 404 returned error can't find the container with id 793bfd5dedbfaa8854595230090113081c176d62a52273c15b8fb5c04b10ae14
Jul 15 19:27:03.649596 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:03.649562    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p" event={"ID":"89a2ec48-4d06-45f9-8cbd-a30144708001","Type":"ContainerStarted","Data":"793bfd5dedbfaa8854595230090113081c176d62a52273c15b8fb5c04b10ae14"}
Jul 15 19:27:04.655933 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:04.655884    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p" event={"ID":"89a2ec48-4d06-45f9-8cbd-a30144708001","Type":"ContainerStarted","Data":"a45dd7a16863b73ddb6e47a981013729c347bf3d3b16023b0faefbec021f1160"}
Jul 15 19:27:04.674695 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:04.674648    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p" podStartSLOduration=1.453828706 podStartE2EDuration="1.67463044s" totalImagesPullingTime="220.801734ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:27:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:27:04.672417659 +0000 UTC m=+696.742277056" watchObservedRunningTime="2026-07-15 19:27:04.67463044 +0000 UTC m=+696.744489838"
Jul 15 19:27:13.449995 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:13.449955    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:13.450495 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:13.450020    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:13.462574 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:13.462549    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:13.705592 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:13.705507    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nx8p"
Jul 15 19:27:15.948635 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:15.948606    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:27:23.347196 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:23.347121    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:27:23.402947 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:23.402915    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:27:26.436590 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.436554    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"]
Jul 15 19:27:26.439948 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.439930    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.442751 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.442726    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 15 19:27:26.449163 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.449141    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"]
Jul 15 19:27:26.452577 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.452547    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:27:26.570099 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.570064    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.570270 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.570107    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.570270 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.570141    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7sx4\" (UniqueName: \"kubernetes.io/projected/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-kube-api-access-q7sx4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.570270 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.570220    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.570270 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.570247    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.570270 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.570263    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.670842 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.670798    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.671028 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.670879    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.671028 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.670915    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.671028 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.670948    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.671028 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.670996    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.671263 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.671036    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7sx4\" (UniqueName: \"kubernetes.io/projected/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-kube-api-access-q7sx4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.671315 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.671286    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.671497 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.671440    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.671636 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.671508    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.672823 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.672798    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.673567 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.673546    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.678836 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.678815    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7sx4\" (UniqueName: \"kubernetes.io/projected/1f1c87d3-9a8e-46df-aa1e-b0908b77650c-kube-api-access-q7sx4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-trkj6\" (UID: \"1f1c87d3-9a8e-46df-aa1e-b0908b77650c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.751882 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.751847    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:26.887121 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.887098    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"]
Jul 15 19:27:26.892714 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:26.892689    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 19:27:27.768687 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:27.768651    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6" event={"ID":"1f1c87d3-9a8e-46df-aa1e-b0908b77650c","Type":"ContainerStarted","Data":"66083a85f295c3d62e211d8ea41f1cb11e73dcb48fe1125f077e88b509cad9c2"}
Jul 15 19:27:27.769069 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:27.768705    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6" event={"ID":"1f1c87d3-9a8e-46df-aa1e-b0908b77650c","Type":"ContainerStarted","Data":"9c87816d99ae8af52fc82a2aa22395e7905d2da7bada42078a1c1b9137e7e297"}
Jul 15 19:27:27.789890 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:27.789827    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6" podStartSLOduration=1.5850013889999999 podStartE2EDuration="1.789809714s" totalImagesPullingTime="204.808325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:27:27.78851482 +0000 UTC m=+719.858374220" watchObservedRunningTime="2026-07-15 19:27:27.789809714 +0000 UTC m=+719.859669111"
Jul 15 19:27:30.444827 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:30.444795    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:27:31.442344 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:31.442310    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:27:36.752118 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:36.752087    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:36.752604 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:36.752150    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:36.764018 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:36.763996    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:36.823356 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:36.823336    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-trkj6"
Jul 15 19:27:40.508576 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:27:40.508545    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:28:48.755654 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:48.755624    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:28:48.820971 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:48.820939    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:28:49.946254 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:49.946209    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:28:50.045843 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:50.045810    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:28:51.322972 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:51.322945    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6796f97bff-8m6p8"]
Jul 15 19:28:51.323422 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:51.323106    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6796f97bff-8m6p8" podUID="5b89a1f6-cfbf-4409-bcdd-8e661bc88be5" containerName="authorino" containerID="cri-o://6a322232474f491ffe79bee0ce727f6b11e1dbd75442724fcb6d2c870842ac26" gracePeriod=30
Jul 15 19:28:51.563869 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:51.563848    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6796f97bff-8m6p8"
Jul 15 19:28:51.608344 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:51.608277    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5b89a1f6-cfbf-4409-bcdd-8e661bc88be5-tls-cert\") pod \"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5\" (UID: \"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5\") "
Jul 15 19:28:51.608510 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:51.608380    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx4cl\" (UniqueName: \"kubernetes.io/projected/5b89a1f6-cfbf-4409-bcdd-8e661bc88be5-kube-api-access-hx4cl\") pod \"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5\" (UID: \"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5\") "
Jul 15 19:28:51.610290 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:51.610257    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b89a1f6-cfbf-4409-bcdd-8e661bc88be5-kube-api-access-hx4cl" (OuterVolumeSpecName: "kube-api-access-hx4cl") pod "5b89a1f6-cfbf-4409-bcdd-8e661bc88be5" (UID: "5b89a1f6-cfbf-4409-bcdd-8e661bc88be5"). InnerVolumeSpecName "kube-api-access-hx4cl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:28:51.618217 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:51.618191    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b89a1f6-cfbf-4409-bcdd-8e661bc88be5-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5b89a1f6-cfbf-4409-bcdd-8e661bc88be5" (UID: "5b89a1f6-cfbf-4409-bcdd-8e661bc88be5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:28:51.709051 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:51.709029    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx4cl\" (UniqueName: \"kubernetes.io/projected/5b89a1f6-cfbf-4409-bcdd-8e661bc88be5-kube-api-access-hx4cl\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:28:51.709051 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:51.709051    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5b89a1f6-cfbf-4409-bcdd-8e661bc88be5-tls-cert\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:28:52.156336 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:52.156303    2560 generic.go:356] "Generic (PLEG): container finished" podID="5b89a1f6-cfbf-4409-bcdd-8e661bc88be5" containerID="6a322232474f491ffe79bee0ce727f6b11e1dbd75442724fcb6d2c870842ac26" exitCode=0
Jul 15 19:28:52.156507 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:52.156347    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6796f97bff-8m6p8"
Jul 15 19:28:52.156507 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:52.156390    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6796f97bff-8m6p8" event={"ID":"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5","Type":"ContainerDied","Data":"6a322232474f491ffe79bee0ce727f6b11e1dbd75442724fcb6d2c870842ac26"}
Jul 15 19:28:52.156507 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:52.156427    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6796f97bff-8m6p8" event={"ID":"5b89a1f6-cfbf-4409-bcdd-8e661bc88be5","Type":"ContainerDied","Data":"6da1b22a996c7ddbf6710c94c097375ca3144faf87db3c7fd08a3e469b6a7b88"}
Jul 15 19:28:52.156507 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:52.156445    2560 scope.go:117] "RemoveContainer" containerID="6a322232474f491ffe79bee0ce727f6b11e1dbd75442724fcb6d2c870842ac26"
Jul 15 19:28:52.165045 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:52.165030    2560 scope.go:117] "RemoveContainer" containerID="6a322232474f491ffe79bee0ce727f6b11e1dbd75442724fcb6d2c870842ac26"
Jul 15 19:28:52.165283 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:28:52.165267    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a322232474f491ffe79bee0ce727f6b11e1dbd75442724fcb6d2c870842ac26\": container with ID starting with 6a322232474f491ffe79bee0ce727f6b11e1dbd75442724fcb6d2c870842ac26 not found: ID does not exist" containerID="6a322232474f491ffe79bee0ce727f6b11e1dbd75442724fcb6d2c870842ac26"
Jul 15 19:28:52.165326 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:52.165293    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a322232474f491ffe79bee0ce727f6b11e1dbd75442724fcb6d2c870842ac26"} err="failed to get container status \"6a322232474f491ffe79bee0ce727f6b11e1dbd75442724fcb6d2c870842ac26\": rpc error: code = NotFound desc = could not find container \"6a322232474f491ffe79bee0ce727f6b11e1dbd75442724fcb6d2c870842ac26\": container with ID starting with 6a322232474f491ffe79bee0ce727f6b11e1dbd75442724fcb6d2c870842ac26 not found: ID does not exist"
Jul 15 19:28:52.176296 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:52.176268    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6796f97bff-8m6p8"]
Jul 15 19:28:52.181995 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:52.181970    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6796f97bff-8m6p8"]
Jul 15 19:28:52.500242 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:28:52.500209    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b89a1f6-cfbf-4409-bcdd-8e661bc88be5" path="/var/lib/kubelet/pods/5b89a1f6-cfbf-4409-bcdd-8e661bc88be5/volumes"
Jul 15 19:29:45.733003 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:29:45.732966    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:29:45.833000 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:29:45.832965    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:00.746289 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:30:00.746258    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:02.403881 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:30:02.403852    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:28.407656 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:30:28.407589    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:30:28.413719 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:30:28.413691    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:30:43.869345 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:30:43.869304    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:48.440368 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:30:48.440337    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:49.084756 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:30:49.084717    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:49.201729 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:30:49.201683    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:49.287998 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:30:49.287963    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:49.444945 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:30:49.444833    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:55.178764 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:30:55.178734    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:30:55.248498 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:30:55.248474    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:31:58.758799 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:31:58.758703    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:31:58.856075 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:31:58.856046    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:32:03.314281 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:32:03.314234    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:33:32.724339 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:33:32.724308    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:34:49.544578 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:49.544548    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"]
Jul 15 19:34:49.545099 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:49.544742    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk" podUID="dd753323-43e7-4d08-b740-00e3386448d8" containerName="manager" containerID="cri-o://d2323c743b3f748a152ebd17a3fe0a0fe42127c06fd8f08b1dca67b21a14f653" gracePeriod=10
Jul 15 19:34:51.795641 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:51.795608    2560 generic.go:356] "Generic (PLEG): container finished" podID="dd753323-43e7-4d08-b740-00e3386448d8" containerID="d2323c743b3f748a152ebd17a3fe0a0fe42127c06fd8f08b1dca67b21a14f653" exitCode=0
Jul 15 19:34:51.795997 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:51.795680    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk" event={"ID":"dd753323-43e7-4d08-b740-00e3386448d8","Type":"ContainerDied","Data":"d2323c743b3f748a152ebd17a3fe0a0fe42127c06fd8f08b1dca67b21a14f653"}
Jul 15 19:34:52.093999 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:52.093974    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"
Jul 15 19:34:52.186815 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:52.186784    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd753323-43e7-4d08-b740-00e3386448d8-extensions-socket-volume\") pod \"dd753323-43e7-4d08-b740-00e3386448d8\" (UID: \"dd753323-43e7-4d08-b740-00e3386448d8\") "
Jul 15 19:34:52.187026 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:52.186865    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ljl5\" (UniqueName: \"kubernetes.io/projected/dd753323-43e7-4d08-b740-00e3386448d8-kube-api-access-9ljl5\") pod \"dd753323-43e7-4d08-b740-00e3386448d8\" (UID: \"dd753323-43e7-4d08-b740-00e3386448d8\") "
Jul 15 19:34:52.187169 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:52.187144    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd753323-43e7-4d08-b740-00e3386448d8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dd753323-43e7-4d08-b740-00e3386448d8" (UID: "dd753323-43e7-4d08-b740-00e3386448d8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:34:52.188825 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:52.188803    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd753323-43e7-4d08-b740-00e3386448d8-kube-api-access-9ljl5" (OuterVolumeSpecName: "kube-api-access-9ljl5") pod "dd753323-43e7-4d08-b740-00e3386448d8" (UID: "dd753323-43e7-4d08-b740-00e3386448d8"). InnerVolumeSpecName "kube-api-access-9ljl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:34:52.287713 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:52.287673    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd753323-43e7-4d08-b740-00e3386448d8-extensions-socket-volume\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:34:52.287713 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:52.287705    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ljl5\" (UniqueName: \"kubernetes.io/projected/dd753323-43e7-4d08-b740-00e3386448d8-kube-api-access-9ljl5\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:34:52.801637 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:52.801560    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"
Jul 15 19:34:52.801637 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:52.801591    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk" event={"ID":"dd753323-43e7-4d08-b740-00e3386448d8","Type":"ContainerDied","Data":"c314fbefa81d10b3e90da7462cb90d452a7d1fc5abe5545402f50be6349a557b"}
Jul 15 19:34:52.802076 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:52.801650    2560 scope.go:117] "RemoveContainer" containerID="d2323c743b3f748a152ebd17a3fe0a0fe42127c06fd8f08b1dca67b21a14f653"
Jul 15 19:34:52.819495 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:52.819473    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"]
Jul 15 19:34:52.822957 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:52.822931    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xqwzk"]
Jul 15 19:34:54.494412 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:34:54.494383    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd753323-43e7-4d08-b740-00e3386448d8" path="/var/lib/kubelet/pods/dd753323-43e7-4d08-b740-00e3386448d8/volumes"
Jul 15 19:35:16.288295 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:16.288264    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:16.372921 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:16.372892    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:35:28.438903 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:28.438874    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:35:28.440653 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:28.440628    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:35:50.746766 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.746737    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx"]
Jul 15 19:35:50.749114 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.747076    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b89a1f6-cfbf-4409-bcdd-8e661bc88be5" containerName="authorino"
Jul 15 19:35:50.749114 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.747087    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b89a1f6-cfbf-4409-bcdd-8e661bc88be5" containerName="authorino"
Jul 15 19:35:50.749114 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.747101    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd753323-43e7-4d08-b740-00e3386448d8" containerName="manager"
Jul 15 19:35:50.749114 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.747106    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd753323-43e7-4d08-b740-00e3386448d8" containerName="manager"
Jul 15 19:35:50.749114 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.747163    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b89a1f6-cfbf-4409-bcdd-8e661bc88be5" containerName="authorino"
Jul 15 19:35:50.749114 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.747172    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd753323-43e7-4d08-b740-00e3386448d8" containerName="manager"
Jul 15 19:35:50.749934 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.749919    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx"
Jul 15 19:35:50.752924 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.752906    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 19:35:50.753035 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.752970    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 19:35:50.753035 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.752975    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-288zc\""
Jul 15 19:35:50.761990 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.761972    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx"]
Jul 15 19:35:50.795048 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.795027    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmgv6\" (UniqueName: \"kubernetes.io/projected/29b660a7-e824-4e1b-a142-c8b6b47df2a9-kube-api-access-pmgv6\") pod \"kuadrant-operator-controller-manager-86878c769f-bkkwx\" (UID: \"29b660a7-e824-4e1b-a142-c8b6b47df2a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx"
Jul 15 19:35:50.795139 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.795112    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29b660a7-e824-4e1b-a142-c8b6b47df2a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bkkwx\" (UID: \"29b660a7-e824-4e1b-a142-c8b6b47df2a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx"
Jul 15 19:35:50.896488 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.896447    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29b660a7-e824-4e1b-a142-c8b6b47df2a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bkkwx\" (UID: \"29b660a7-e824-4e1b-a142-c8b6b47df2a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx"
Jul 15 19:35:50.896573 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.896503    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmgv6\" (UniqueName: \"kubernetes.io/projected/29b660a7-e824-4e1b-a142-c8b6b47df2a9-kube-api-access-pmgv6\") pod \"kuadrant-operator-controller-manager-86878c769f-bkkwx\" (UID: \"29b660a7-e824-4e1b-a142-c8b6b47df2a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx"
Jul 15 19:35:50.896758 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.896743    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29b660a7-e824-4e1b-a142-c8b6b47df2a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bkkwx\" (UID: \"29b660a7-e824-4e1b-a142-c8b6b47df2a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx"
Jul 15 19:35:50.905261 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:50.905234    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmgv6\" (UniqueName: \"kubernetes.io/projected/29b660a7-e824-4e1b-a142-c8b6b47df2a9-kube-api-access-pmgv6\") pod \"kuadrant-operator-controller-manager-86878c769f-bkkwx\" (UID: \"29b660a7-e824-4e1b-a142-c8b6b47df2a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx"
Jul 15 19:35:51.059409 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:51.059341    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx"
Jul 15 19:35:51.185875 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:51.185847    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx"]
Jul 15 19:35:51.187043 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:35:51.187015    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b660a7_e824_4e1b_a142_c8b6b47df2a9.slice/crio-1f8e804f0b0ccb12a342ba739f89746ffda706490e5c205f494cf55cd150628d WatchSource:0}: Error finding container 1f8e804f0b0ccb12a342ba739f89746ffda706490e5c205f494cf55cd150628d: Status 404 returned error can't find the container with id 1f8e804f0b0ccb12a342ba739f89746ffda706490e5c205f494cf55cd150628d
Jul 15 19:35:52.072204 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:52.072168    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx" event={"ID":"29b660a7-e824-4e1b-a142-c8b6b47df2a9","Type":"ContainerStarted","Data":"30069839ca68d74fde01ea2610444ef86baf7892a8aba77dbf7a1771db64c839"}
Jul 15 19:35:52.072666 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:52.072241    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx" event={"ID":"29b660a7-e824-4e1b-a142-c8b6b47df2a9","Type":"ContainerStarted","Data":"1f8e804f0b0ccb12a342ba739f89746ffda706490e5c205f494cf55cd150628d"}
Jul 15 19:35:52.072666 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:52.072262    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx"
Jul 15 19:35:52.108587 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:35:52.108527    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx" podStartSLOduration=2.108513223 podStartE2EDuration="2.108513223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:35:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:35:52.1059163 +0000 UTC m=+1224.175775696" watchObservedRunningTime="2026-07-15 19:35:52.108513223 +0000 UTC m=+1224.178372621"
Jul 15 19:36:03.078857 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:36:03.078829    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bkkwx"
Jul 15 19:40:06.146531 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:40:06.146493    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:40:08.107204 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:40:08.107165    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:40:28.468684 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:40:28.468657    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:40:28.469195 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:40:28.468954    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:40:43.267060 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:40:43.267025    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:41:33.526471 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:41:33.526418    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:41:41.045418 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:41:41.045382    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:41:49.406131 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:41:49.406098    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:41:54.472320 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:41:54.472284    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:41:57.637188 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:41:57.637149    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:41:57.735624 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:41:57.735594    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:41:58.102192 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:41:58.102161    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:03.861134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:42:03.861093    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:03.962514 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:42:03.962440    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:04.080816 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:42:04.080780    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:04.144896 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:42:04.144816    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:05.189008 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:42:05.188977    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:05.290357 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:42:05.290312    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:05.383273 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:42:05.383240    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:05.483610 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:42:05.483576    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:05.861384 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:42:05.861271    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:05.961843 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:42:05.961810    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:07.986241 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:42:07.986208    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:42:46.249355 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:42:46.249319    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:10.788104 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:10.788050    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:15.470785 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:15.470744    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:15.749742 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:15.749709    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:15.940210 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:15.940174    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:21.679527 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:21.679491    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:21.781406 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:21.781373    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:21.962908 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:21.962825    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:22.058745 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:22.058712    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:22.162812 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:22.162779    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:22.564860 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:22.564830    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:22.663339 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:22.663294    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:23.374167 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:23.374127    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:23.557502 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:23.557469    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:23.987515 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:23.987484    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:24.089243 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:24.089214    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:24.204736 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:24.204698    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:43:25.792774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:43:25.792743    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:28.096366 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:28.096328    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:28.195176 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:28.195137    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:28.248936 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:28.248904    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:33.582610 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:33.582555    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:34.255720 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:34.255686    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:34.727893 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:34.727861    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:39.812893 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:39.812855    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:39.882536 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:39.882509    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:40.057997 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:40.057966    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:40.117680 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:40.117616    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:41.157842 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:41.157806    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:41.235826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:41.235773    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:41.311657 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:41.311622    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:41.565885 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:41.565840    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:41.807797 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:41.807764    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:41.906205 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:41.906129    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:44:42.992872 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:44:42.992837    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:25.505337 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:25.505254    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:26.676788 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:26.676756    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:28.500269 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:28.500239    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:45:28.500708 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:28.500634    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:45:37.266911 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:37.266871    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:37.366957 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:37.366926    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:47.005311 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:47.005265    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:51.679695 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:51.679640    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:52.970747 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:52.970712    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:53.078151 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:53.078110    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:58.845415 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:58.845376    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:58.946599 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:58.946565    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:59.064362 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:59.064330    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:59.421028 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:59.420995    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:45:59.480803 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:45:59.480755    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:46:00.203326 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:46:00.203295    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:46:00.291433 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:46:00.291398    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:46:00.362445 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:46:00.362403    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:46:00.480442 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:46:00.480349    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:46:00.829206 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:46:00.829170    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:46:00.930083 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:46:00.930057    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:46:03.175958 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:46:03.175923    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:46:50.100592 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:46:50.100511    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:05.161727 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:05.161684    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:09.723249 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:09.723215    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:09.896135 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:09.896094    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:10.001309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:10.001277    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:15.980916 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:15.980880    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:16.085011 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:16.084975    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:16.369048 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:16.369004    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:16.504377 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:16.504350    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:16.604513 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:16.604401    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:16.705256 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:16.705215    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:18.286737 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:18.286701    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:24.394929 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:24.394883    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:24.512397 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:24.512362    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:24.598050 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:24.597998    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:24.699000 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:24.698908    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:25.138038 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:25.137998    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:51.142800 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:51.142762    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:57.260227 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:57.260194    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:57.365840 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:57.365807    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:47:57.461004 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:47:57.460966    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:13.410244 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:13.410205    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:19.150034 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:19.150003    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:19.260840 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:19.260803    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:19.350815 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:19.350779    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:24.137808 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:24.137723    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:29.953717 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:29.953682    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:30.781986 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:30.781923    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:37.036212 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:37.036160    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:37.142049 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:37.142019    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:37.238004 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:37.237967    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:37.338328 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:37.338238    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:38.206272 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:38.206206    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:38.305580 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:38.305544    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:38.433679 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:38.433643    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:40.149391 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:40.149359    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:45.828593 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:45.828561    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:45.901424 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:45.901390    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:46.540849 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:46.540813    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:52.234590 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:52.234555    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:52.846196 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:52.846164    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:53.369693 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:53.369654    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:53.470853 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:53.470827    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:56.159186 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:56.159143    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:56.248864 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:56.248836    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:56.318329 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:56.318298    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:56.667527 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:56.667494    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:56.790120 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:56.790083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:48:56.894570 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:48:56.894535    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:00.708747 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:00.708707    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:01.430325 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:01.430292    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:01.524045 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:01.524010    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:01.914134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:01.914098    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:07.767955 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:07.767919    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:07.830397 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:07.830366    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:08.173999 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:08.173910    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg"]
Jul 15 19:49:08.177365 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:08.177347    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg"
Jul 15 19:49:08.179865 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:08.179841    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 19:49:08.179983 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:08.179847    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bk6kr\""
Jul 15 19:49:08.185527 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:08.185493    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg"]
Jul 15 19:49:08.297378 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:08.297345    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/beadb106-5009-4a80-b208-3be02f16f80c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg\" (UID: \"beadb106-5009-4a80-b208-3be02f16f80c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg"
Jul 15 19:49:08.398890 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:08.398838    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/beadb106-5009-4a80-b208-3be02f16f80c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg\" (UID: \"beadb106-5009-4a80-b208-3be02f16f80c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg"
Jul 15 19:49:08.399521 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:08.399503    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/beadb106-5009-4a80-b208-3be02f16f80c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg\" (UID: \"beadb106-5009-4a80-b208-3be02f16f80c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg"
Jul 15 19:49:08.489882 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:08.489845    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg"
Jul 15 19:49:08.622096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:08.622070    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg"]
Jul 15 19:49:08.624050 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:49:08.624012    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeadb106_5009_4a80_b208_3be02f16f80c.slice/crio-81b8424e18a7ee10c01ced7b36e7736ca97303be4bb112556a897a75cbf8ee2c WatchSource:0}: Error finding container 81b8424e18a7ee10c01ced7b36e7736ca97303be4bb112556a897a75cbf8ee2c: Status 404 returned error can't find the container with id 81b8424e18a7ee10c01ced7b36e7736ca97303be4bb112556a897a75cbf8ee2c
Jul 15 19:49:08.626100 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:08.626084    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 19:49:08.659761 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:08.659728    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg" event={"ID":"beadb106-5009-4a80-b208-3be02f16f80c","Type":"ContainerStarted","Data":"81b8424e18a7ee10c01ced7b36e7736ca97303be4bb112556a897a75cbf8ee2c"}
Jul 15 19:49:11.676417 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:11.676323    2560 generic.go:356] "Generic (PLEG): container finished" podID="beadb106-5009-4a80-b208-3be02f16f80c" containerID="bbd8d909719c0faef4b7b1758b7fc8f1ce4be61b8564fd3f1baa3823a7c72f8c" exitCode=0
Jul 15 19:49:11.676788 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:11.676408    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg" event={"ID":"beadb106-5009-4a80-b208-3be02f16f80c","Type":"ContainerDied","Data":"bbd8d909719c0faef4b7b1758b7fc8f1ce4be61b8564fd3f1baa3823a7c72f8c"}
Jul 15 19:49:12.811196 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:12.811171    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg"
Jul 15 19:49:12.942054 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:12.941957    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/beadb106-5009-4a80-b208-3be02f16f80c-maas-api-service-ca\") pod \"beadb106-5009-4a80-b208-3be02f16f80c\" (UID: \"beadb106-5009-4a80-b208-3be02f16f80c\") "
Jul 15 19:49:12.942329 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:12.942306    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beadb106-5009-4a80-b208-3be02f16f80c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "beadb106-5009-4a80-b208-3be02f16f80c" (UID: "beadb106-5009-4a80-b208-3be02f16f80c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:49:13.043116 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:13.043084    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/beadb106-5009-4a80-b208-3be02f16f80c-maas-api-service-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:49:13.686587 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:13.686551    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg" event={"ID":"beadb106-5009-4a80-b208-3be02f16f80c","Type":"ContainerDied","Data":"81b8424e18a7ee10c01ced7b36e7736ca97303be4bb112556a897a75cbf8ee2c"}
Jul 15 19:49:13.686788 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:13.686605    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81b8424e18a7ee10c01ced7b36e7736ca97303be4bb112556a897a75cbf8ee2c"
Jul 15 19:49:13.686788 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:13.686559    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg"
Jul 15 19:49:18.600100 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:18.600066    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:23.255354 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:23.255317    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:23.675825 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:23.675748    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:29.439000 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:29.438962    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:29.508701 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:29.508651    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:29.730037 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:29.729955    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:29.958833 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:29.958789    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:30.095896 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:30.095839    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:30.187366 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:30.187298    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:30.328423 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:30.328389    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:36.048357 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:36.048323    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:36.157889 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:36.157849    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:36.248731 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:36.248698    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:36.413768 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:36.413671    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:42.053902 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:42.053864    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:42.153899 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:42.153864    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:46.747820 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:46.747786    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:46.802510 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:46.802471    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:49.047240 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:49.047205    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:49.403189 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:49.403108    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:49.806390 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:49.806352    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:52.406738 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:52.406694    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:52.496444 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:52.496408    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:52.567582 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:52.567551    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:52.781324 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:52.781295    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:52.965500 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:52.965467    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:53.066007 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:53.065926    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:49:56.963413 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:49:56.963374    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:50:17.943950 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:17.943916    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"]
Jul 15 19:50:17.944344 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:17.944327    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beadb106-5009-4a80-b208-3be02f16f80c" containerName="revoke-keys"
Jul 15 19:50:17.944344 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:17.944339    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="beadb106-5009-4a80-b208-3be02f16f80c" containerName="revoke-keys"
Jul 15 19:50:17.944417 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:17.944409    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="beadb106-5009-4a80-b208-3be02f16f80c" containerName="revoke-keys"
Jul 15 19:50:17.947649 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:17.947630    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:17.951564 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:17.951544    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-80adc0ad\"/\"default-dockercfg-gqjz9\""
Jul 15 19:50:17.951686 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:17.951594    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-80adc0ad\"/\"e2e-lifecycle-model-80adc0ad-kserve-self-signed-certs\""
Jul 15 19:50:17.951868 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:17.951849    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-80adc0ad\"/\"kube-root-ca.crt\""
Jul 15 19:50:17.951953 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:17.951875    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-80adc0ad\"/\"openshift-service-ca.crt\""
Jul 15 19:50:17.957315 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:17.957295    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"]
Jul 15 19:50:18.026019 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.025983    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:50:18.105418 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.105386    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhgps\" (UniqueName: \"kubernetes.io/projected/a7dd342f-da24-462b-bca5-88513c695ffc-kube-api-access-lhgps\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.105418 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.105420    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-home\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.105644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.105482    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-model-cache\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.105644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.105507    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-tmp-dir\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.105644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.105577    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-dshm\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.105644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.105612    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7dd342f-da24-462b-bca5-88513c695ffc-tls-certs\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.206619 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.206538    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-dshm\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.206619 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.206587    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7dd342f-da24-462b-bca5-88513c695ffc-tls-certs\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.206619 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.206621    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhgps\" (UniqueName: \"kubernetes.io/projected/a7dd342f-da24-462b-bca5-88513c695ffc-kube-api-access-lhgps\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.206884 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.206642    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-home\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.206884 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.206673    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-model-cache\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.206884 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.206693    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-tmp-dir\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.207065 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.207028    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-home\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.207127 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.207077    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-tmp-dir\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.207176 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.207126    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-model-cache\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.208783 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.208763    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-dshm\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.209040 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.209021    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7dd342f-da24-462b-bca5-88513c695ffc-tls-certs\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.215209 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.215188    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhgps\" (UniqueName: \"kubernetes.io/projected/a7dd342f-da24-462b-bca5-88513c695ffc-kube-api-access-lhgps\") pod \"e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") " pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.265782 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.265760    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:18.393696 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.393673    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"]
Jul 15 19:50:18.396027 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:50:18.395998    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7dd342f_da24_462b_bca5_88513c695ffc.slice/crio-14e9c2676d259236b8b52503f86be7df97e83edaad66f9a7bb2dbcfa1ed6dff4 WatchSource:0}: Error finding container 14e9c2676d259236b8b52503f86be7df97e83edaad66f9a7bb2dbcfa1ed6dff4: Status 404 returned error can't find the container with id 14e9c2676d259236b8b52503f86be7df97e83edaad66f9a7bb2dbcfa1ed6dff4
Jul 15 19:50:18.987939 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.987908    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x" event={"ID":"a7dd342f-da24-462b-bca5-88513c695ffc","Type":"ContainerStarted","Data":"8f780b64d7d078cc0886d02bc545b716a3599e8029d8f8c3bec01407a54f7491"}
Jul 15 19:50:18.987939 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:18.987947    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x" event={"ID":"a7dd342f-da24-462b-bca5-88513c695ffc","Type":"ContainerStarted","Data":"14e9c2676d259236b8b52503f86be7df97e83edaad66f9a7bb2dbcfa1ed6dff4"}
Jul 15 19:50:19.005615 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:19.005572    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x" podStartSLOduration=2.005552982 podStartE2EDuration="2.005552982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:50:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:50:19.004768642 +0000 UTC m=+2091.074628039" watchObservedRunningTime="2026-07-15 19:50:19.005552982 +0000 UTC m=+2091.075412381"
Jul 15 19:50:26.286493 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:26.286437    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:50:26.491341 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:26.491285    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:50:28.266087 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:28.266040    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:28.266532 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:28.266299    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:28.278466 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:28.278432    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:28.526245 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:28.526165    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:50:28.528414 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:28.528395    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:50:29.045280 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:29.045250    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:50:30.667520 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:30.667484    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:50:32.432247 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:32.432208    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:50:51.637032 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:51.637000    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:50:51.742634 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:50:51.742597    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:51:03.564262 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:03.564225    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:51:06.449771 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:06.449734    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:51:11.123139 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:11.123096    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:51:13.455510 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:13.455472    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:51:18.445622 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:18.445572    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"]
Jul 15 19:51:18.540026 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:18.539982    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:51:33.465692 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.465626    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x" podUID="a7dd342f-da24-462b-bca5-88513c695ffc" containerName="main" containerID="cri-o://8f780b64d7d078cc0886d02bc545b716a3599e8029d8f8c3bec01407a54f7491" gracePeriod=45
Jul 15 19:51:33.710953 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.710923    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:51:33.790734 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.790696    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7dd342f-da24-462b-bca5-88513c695ffc-tls-certs\") pod \"a7dd342f-da24-462b-bca5-88513c695ffc\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") "
Jul 15 19:51:33.790967 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.790777    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhgps\" (UniqueName: \"kubernetes.io/projected/a7dd342f-da24-462b-bca5-88513c695ffc-kube-api-access-lhgps\") pod \"a7dd342f-da24-462b-bca5-88513c695ffc\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") "
Jul 15 19:51:33.790967 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.790826    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-model-cache\") pod \"a7dd342f-da24-462b-bca5-88513c695ffc\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") "
Jul 15 19:51:33.790967 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.790853    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-dshm\") pod \"a7dd342f-da24-462b-bca5-88513c695ffc\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") "
Jul 15 19:51:33.790967 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.790888    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-tmp-dir\") pod \"a7dd342f-da24-462b-bca5-88513c695ffc\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") "
Jul 15 19:51:33.790967 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.790909    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-home\") pod \"a7dd342f-da24-462b-bca5-88513c695ffc\" (UID: \"a7dd342f-da24-462b-bca5-88513c695ffc\") "
Jul 15 19:51:33.791251 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.791131    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a7dd342f-da24-462b-bca5-88513c695ffc" (UID: "a7dd342f-da24-462b-bca5-88513c695ffc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:51:33.791251 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.791145    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-model-cache" (OuterVolumeSpecName: "model-cache") pod "a7dd342f-da24-462b-bca5-88513c695ffc" (UID: "a7dd342f-da24-462b-bca5-88513c695ffc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:51:33.791251 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.791216    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-home" (OuterVolumeSpecName: "home") pod "a7dd342f-da24-462b-bca5-88513c695ffc" (UID: "a7dd342f-da24-462b-bca5-88513c695ffc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:51:33.792995 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.792969    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7dd342f-da24-462b-bca5-88513c695ffc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a7dd342f-da24-462b-bca5-88513c695ffc" (UID: "a7dd342f-da24-462b-bca5-88513c695ffc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:51:33.793107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.793011    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7dd342f-da24-462b-bca5-88513c695ffc-kube-api-access-lhgps" (OuterVolumeSpecName: "kube-api-access-lhgps") pod "a7dd342f-da24-462b-bca5-88513c695ffc" (UID: "a7dd342f-da24-462b-bca5-88513c695ffc"). InnerVolumeSpecName "kube-api-access-lhgps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:51:33.793107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.793037    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-dshm" (OuterVolumeSpecName: "dshm") pod "a7dd342f-da24-462b-bca5-88513c695ffc" (UID: "a7dd342f-da24-462b-bca5-88513c695ffc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:51:33.892110 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.892061    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhgps\" (UniqueName: \"kubernetes.io/projected/a7dd342f-da24-462b-bca5-88513c695ffc-kube-api-access-lhgps\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:51:33.892110 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.892104    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-model-cache\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:51:33.892110 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.892114    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-dshm\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:51:33.892110 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.892121    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-tmp-dir\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:51:33.892110 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.892129    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7dd342f-da24-462b-bca5-88513c695ffc-home\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:51:33.892445 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:33.892138    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7dd342f-da24-462b-bca5-88513c695ffc-tls-certs\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:51:34.344241 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:34.344212    2560 generic.go:356] "Generic (PLEG): container finished" podID="a7dd342f-da24-462b-bca5-88513c695ffc" containerID="8f780b64d7d078cc0886d02bc545b716a3599e8029d8f8c3bec01407a54f7491" exitCode=0
Jul 15 19:51:34.344424 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:34.344252    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"
Jul 15 19:51:34.344424 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:34.344292    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x" event={"ID":"a7dd342f-da24-462b-bca5-88513c695ffc","Type":"ContainerDied","Data":"8f780b64d7d078cc0886d02bc545b716a3599e8029d8f8c3bec01407a54f7491"}
Jul 15 19:51:34.344424 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:34.344335    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x" event={"ID":"a7dd342f-da24-462b-bca5-88513c695ffc","Type":"ContainerDied","Data":"14e9c2676d259236b8b52503f86be7df97e83edaad66f9a7bb2dbcfa1ed6dff4"}
Jul 15 19:51:34.344424 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:34.344352    2560 scope.go:117] "RemoveContainer" containerID="8f780b64d7d078cc0886d02bc545b716a3599e8029d8f8c3bec01407a54f7491"
Jul 15 19:51:34.353148 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:34.353130    2560 scope.go:117] "RemoveContainer" containerID="8f780b64d7d078cc0886d02bc545b716a3599e8029d8f8c3bec01407a54f7491"
Jul 15 19:51:34.353405 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:51:34.353386    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f780b64d7d078cc0886d02bc545b716a3599e8029d8f8c3bec01407a54f7491\": container with ID starting with 8f780b64d7d078cc0886d02bc545b716a3599e8029d8f8c3bec01407a54f7491 not found: ID does not exist" containerID="8f780b64d7d078cc0886d02bc545b716a3599e8029d8f8c3bec01407a54f7491"
Jul 15 19:51:34.353484 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:34.353412    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f780b64d7d078cc0886d02bc545b716a3599e8029d8f8c3bec01407a54f7491"} err="failed to get container status \"8f780b64d7d078cc0886d02bc545b716a3599e8029d8f8c3bec01407a54f7491\": rpc error: code = NotFound desc = could not find container \"8f780b64d7d078cc0886d02bc545b716a3599e8029d8f8c3bec01407a54f7491\": container with ID starting with 8f780b64d7d078cc0886d02bc545b716a3599e8029d8f8c3bec01407a54f7491 not found: ID does not exist"
Jul 15 19:51:34.364462 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:34.364431    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"]
Jul 15 19:51:34.367223 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:34.367201    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-80adc0ad/e2e-lifecycle-model-80adc0ad-kserve-768556fb5f-l5s8x"]
Jul 15 19:51:34.495337 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:34.495302    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7dd342f-da24-462b-bca5-88513c695ffc" path="/var/lib/kubelet/pods/a7dd342f-da24-462b-bca5-88513c695ffc/volumes"
Jul 15 19:51:55.430273 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:55.430240    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:51:55.571471 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:51:55.571417    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:01.006791 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:01.006761    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:06.580117 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:06.580053    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:06.647885 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:06.647847    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:06.879637 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:06.879536    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:07.131011 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:07.130908    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:08.278375 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:08.278341    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:08.365938 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:08.365905    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:08.462946 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:08.462907    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:09.321690 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:09.321654    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:09.412045 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:09.412010    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:09.512141 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:09.512103    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:10.981046 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:10.981002    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:16.731826 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:16.731789    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:16.833675 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:16.833642    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:17.468214 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:17.468180    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:23.645707 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:23.645664    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:23.752099 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:23.752067    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:23.844927 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:23.844896    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:34.405817 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:34.405782    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:34.813020 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:34.812986    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:52:55.805138 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:52:55.805053    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:01.898864 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:01.898824    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:01.999034 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:01.998997    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:02.353466 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:02.353412    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:02.612877 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:02.612787    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:02.730432 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:02.730395    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:05.459209 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:05.459168    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:05.559987 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:05.559939    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:05.688301 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:05.688270    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:06.164413 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:06.164375    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:06.342430 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:06.342380    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:06.445157 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:06.445071    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:10.789287 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:10.789144    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:10.889197 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:10.889152    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:11.241699 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:11.241667    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:13.490888 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:13.490847    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:13.588060 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:13.588023    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:13.765835 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:13.765734    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:14.287748 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:14.287661    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:14.375684 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:14.375645    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:14.479010 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:14.478976    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:18.036797 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:18.036754    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:27.106267 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:27.106227    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:27.207411 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:27.207374    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:29.626714 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.626675    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"]
Jul 15 19:53:29.627225 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.627202    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7dd342f-da24-462b-bca5-88513c695ffc" containerName="main"
Jul 15 19:53:29.627312 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.627228    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7dd342f-da24-462b-bca5-88513c695ffc" containerName="main"
Jul 15 19:53:29.627388 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.627323    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7dd342f-da24-462b-bca5-88513c695ffc" containerName="main"
Jul 15 19:53:29.630037 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.630015    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.633993 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.633968    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-da0342\"/\"kube-root-ca.crt\""
Jul 15 19:53:29.635014 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.634992    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-da0342\"/\"openshift-service-ca.crt\""
Jul 15 19:53:29.635192 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.635172    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-da0342\"/\"default-dockercfg-h2hnq\""
Jul 15 19:53:29.635277 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.634998    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-da0342\"/\"test-model-da0342-kserve-self-signed-certs\""
Jul 15 19:53:29.642007 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.641979    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"]
Jul 15 19:53:29.656157 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.656118    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:29.764936 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.764892    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e531e587-6554-4305-864e-bfc34a56dd41-tls-certs\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.764936 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.764941    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-model-cache\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.765212 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.765035    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p629\" (UniqueName: \"kubernetes.io/projected/e531e587-6554-4305-864e-bfc34a56dd41-kube-api-access-9p629\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.765212 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.765109    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-dshm\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.765212 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.765135    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-tmp-dir\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.765348 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.765233    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-home\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.865948 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.865885    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-dshm\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.865948 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.865946    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-tmp-dir\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.866238 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.866002    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-home\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.866238 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.866041    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e531e587-6554-4305-864e-bfc34a56dd41-tls-certs\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.866238 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.866078    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-model-cache\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.866238 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.866119    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p629\" (UniqueName: \"kubernetes.io/projected/e531e587-6554-4305-864e-bfc34a56dd41-kube-api-access-9p629\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.866498 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.866438    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-tmp-dir\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.866578 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.866498    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-home\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.866578 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.866529    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-model-cache\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.868300 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.868281    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-dshm\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.868592 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.868575    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e531e587-6554-4305-864e-bfc34a56dd41-tls-certs\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.875027 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.875005    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p629\" (UniqueName: \"kubernetes.io/projected/e531e587-6554-4305-864e-bfc34a56dd41-kube-api-access-9p629\") pod \"test-model-da0342-kserve-8465ff8df-xddt2\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") " pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:29.954240 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:29.954130    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:30.091662 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:30.091636    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"]
Jul 15 19:53:30.093661 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:53:30.093635    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode531e587_6554_4305_864e_bfc34a56dd41.slice/crio-dc0ad525755ee5cccb9a56b8b040afe4f9857adc4444c4bb71c273de6be0bcb1 WatchSource:0}: Error finding container dc0ad525755ee5cccb9a56b8b040afe4f9857adc4444c4bb71c273de6be0bcb1: Status 404 returned error can't find the container with id dc0ad525755ee5cccb9a56b8b040afe4f9857adc4444c4bb71c273de6be0bcb1
Jul 15 19:53:30.890068 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:30.890028    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2" event={"ID":"e531e587-6554-4305-864e-bfc34a56dd41","Type":"ContainerStarted","Data":"8155848d3ed5fe349d88f6e8f429a0b565bf4ac97fc5961e6e9b33d5775813cc"}
Jul 15 19:53:30.890535 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:30.890081    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2" event={"ID":"e531e587-6554-4305-864e-bfc34a56dd41","Type":"ContainerStarted","Data":"dc0ad525755ee5cccb9a56b8b040afe4f9857adc4444c4bb71c273de6be0bcb1"}
Jul 15 19:53:30.906521 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:30.906426    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2" podStartSLOduration=1.906408591 podStartE2EDuration="1.906408591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:53:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:53:30.905768657 +0000 UTC m=+2282.975628056" watchObservedRunningTime="2026-07-15 19:53:30.906408591 +0000 UTC m=+2282.976268026"
Jul 15 19:53:35.908745 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:35.908704    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:36.008814 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:36.008779    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:39.954812 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:39.954774    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:39.955207 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:39.954826    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:39.967359 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:39.967334    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:40.949000 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:40.948970    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:53:53.245974 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.245928    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"]
Jul 15 19:53:53.249443 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.249415    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.252331 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.252309    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-435e1e\"/\"kube-root-ca.crt\""
Jul 15 19:53:53.253503 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.253482    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-435e1e\"/\"openshift-service-ca.crt\""
Jul 15 19:53:53.253606 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.253587    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-435e1e\"/\"test-model-435e1e-kserve-self-signed-certs\""
Jul 15 19:53:53.253678 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.253606    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-435e1e\"/\"default-dockercfg-rc6p8\""
Jul 15 19:53:53.260960 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.260936    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"]
Jul 15 19:53:53.284467 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.284411    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:53:53.396395 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.396363    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-dshm\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.396579 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.396417    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z55q\" (UniqueName: \"kubernetes.io/projected/ff12b272-fba4-4aae-84fc-66ec2754d1e3-kube-api-access-6z55q\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.396579 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.396515    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-tmp-dir\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.396579 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.396557    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-model-cache\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.396686 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.396581    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-home\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.396686 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.396599    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff12b272-fba4-4aae-84fc-66ec2754d1e3-tls-certs\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.498101 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.498000    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-dshm\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.498101 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.498058    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z55q\" (UniqueName: \"kubernetes.io/projected/ff12b272-fba4-4aae-84fc-66ec2754d1e3-kube-api-access-6z55q\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.498101 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.498090    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-tmp-dir\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.498346 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.498110    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-model-cache\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.498346 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.498125    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-home\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.498346 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.498142    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff12b272-fba4-4aae-84fc-66ec2754d1e3-tls-certs\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.498594 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.498563    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-tmp-dir\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.498695 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.498577    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-model-cache\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.498695 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.498638    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-home\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.500486 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.500461    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-dshm\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.500680 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.500664    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff12b272-fba4-4aae-84fc-66ec2754d1e3-tls-certs\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.506501 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.506475    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z55q\" (UniqueName: \"kubernetes.io/projected/ff12b272-fba4-4aae-84fc-66ec2754d1e3-kube-api-access-6z55q\") pod \"test-model-435e1e-kserve-74cf74cddc-b49cc\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") " pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.561132 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.561097    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:53:53.708064 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:53.707999    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"]
Jul 15 19:53:53.710815 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:53:53.710784    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff12b272_fba4_4aae_84fc_66ec2754d1e3.slice/crio-68630da94e4ef88cd1ff514dc14e2790c47606e4b92dd1c61021fd36f9bf70ba WatchSource:0}: Error finding container 68630da94e4ef88cd1ff514dc14e2790c47606e4b92dd1c61021fd36f9bf70ba: Status 404 returned error can't find the container with id 68630da94e4ef88cd1ff514dc14e2790c47606e4b92dd1c61021fd36f9bf70ba
Jul 15 19:53:54.004897 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:54.004809    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc" event={"ID":"ff12b272-fba4-4aae-84fc-66ec2754d1e3","Type":"ContainerStarted","Data":"465ee3044429d33e067762811eedb7dfb154d29d2d1d07da6ecc0aa4f2b08795"}
Jul 15 19:53:54.004897 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:54.004850    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc" event={"ID":"ff12b272-fba4-4aae-84fc-66ec2754d1e3","Type":"ContainerStarted","Data":"68630da94e4ef88cd1ff514dc14e2790c47606e4b92dd1c61021fd36f9bf70ba"}
Jul 15 19:53:54.024593 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:53:54.024536    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc" podStartSLOduration=1.024516491 podStartE2EDuration="1.024516491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:53:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:53:54.022404793 +0000 UTC m=+2306.092264195" watchObservedRunningTime="2026-07-15 19:53:54.024516491 +0000 UTC m=+2306.094375947"
Jul 15 19:54:03.561470 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:03.561384    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:54:03.561996 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:03.561498    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:54:03.574395 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:03.574364    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:54:04.064984 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:04.064945    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:54:11.337028 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:11.336977    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:11.725117 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:11.725023    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"]
Jul 15 19:54:11.743560 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:11.743526    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:12.653951 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:12.653918    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"]
Jul 15 19:54:12.673079 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:12.673050    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:12.806665 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:12.806625    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d"]
Jul 15 19:54:12.810216 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:12.810200    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d"
Jul 15 19:54:12.813014 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:12.812992    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 19:54:12.813130 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:12.812999    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bk6kr\""
Jul 15 19:54:12.817310 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:12.817287    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d"]
Jul 15 19:54:12.859212 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:12.859182    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/baecea79-9943-4766-9212-aa46ff27f4d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d\" (UID: \"baecea79-9943-4766-9212-aa46ff27f4d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d"
Jul 15 19:54:12.960304 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:12.960235    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/baecea79-9943-4766-9212-aa46ff27f4d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d\" (UID: \"baecea79-9943-4766-9212-aa46ff27f4d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d"
Jul 15 19:54:12.960837 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:12.960819    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/baecea79-9943-4766-9212-aa46ff27f4d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d\" (UID: \"baecea79-9943-4766-9212-aa46ff27f4d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d"
Jul 15 19:54:13.121842 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:13.121807    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d"
Jul 15 19:54:13.251289 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:13.251264    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d"]
Jul 15 19:54:13.253143 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:54:13.253112    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaecea79_9943_4766_9212_aa46ff27f4d9.slice/crio-eb8c8a1901cd3b3106462de14cc7acc20bd01595536eae81ab040b8445b1e113 WatchSource:0}: Error finding container eb8c8a1901cd3b3106462de14cc7acc20bd01595536eae81ab040b8445b1e113: Status 404 returned error can't find the container with id eb8c8a1901cd3b3106462de14cc7acc20bd01595536eae81ab040b8445b1e113
Jul 15 19:54:13.317726 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:13.317700    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:13.419823 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:13.419795    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:14.102399 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:14.102371    2560 generic.go:356] "Generic (PLEG): container finished" podID="baecea79-9943-4766-9212-aa46ff27f4d9" containerID="40f98d98884e5e2eea1a9a1ff6a2acf3cc09280ecf9b682756403bf782d0f364" exitCode=0
Jul 15 19:54:14.102741 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:14.102418    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d" event={"ID":"baecea79-9943-4766-9212-aa46ff27f4d9","Type":"ContainerDied","Data":"40f98d98884e5e2eea1a9a1ff6a2acf3cc09280ecf9b682756403bf782d0f364"}
Jul 15 19:54:14.102741 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:14.102492    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d" event={"ID":"baecea79-9943-4766-9212-aa46ff27f4d9","Type":"ContainerStarted","Data":"eb8c8a1901cd3b3106462de14cc7acc20bd01595536eae81ab040b8445b1e113"}
Jul 15 19:54:15.243493 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:15.243463    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d"
Jul 15 19:54:15.280892 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:15.280866    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/baecea79-9943-4766-9212-aa46ff27f4d9-maas-api-service-ca\") pod \"baecea79-9943-4766-9212-aa46ff27f4d9\" (UID: \"baecea79-9943-4766-9212-aa46ff27f4d9\") "
Jul 15 19:54:15.281197 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:15.281177    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baecea79-9943-4766-9212-aa46ff27f4d9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "baecea79-9943-4766-9212-aa46ff27f4d9" (UID: "baecea79-9943-4766-9212-aa46ff27f4d9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:54:15.381673 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:15.381595    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/baecea79-9943-4766-9212-aa46ff27f4d9-maas-api-service-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:16.113667 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:16.113637    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d"
Jul 15 19:54:16.113667 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:16.113653    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d" event={"ID":"baecea79-9943-4766-9212-aa46ff27f4d9","Type":"ContainerDied","Data":"eb8c8a1901cd3b3106462de14cc7acc20bd01595536eae81ab040b8445b1e113"}
Jul 15 19:54:16.113893 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:16.113704    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb8c8a1901cd3b3106462de14cc7acc20bd01595536eae81ab040b8445b1e113"
Jul 15 19:54:23.240738 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:23.240655    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:26.746248 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:26.746196    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2" podUID="e531e587-6554-4305-864e-bfc34a56dd41" containerName="main" containerID="cri-o://8155848d3ed5fe349d88f6e8f429a0b565bf4ac97fc5961e6e9b33d5775813cc" gracePeriod=45
Jul 15 19:54:26.987885 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:26.987858    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:54:27.079792 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.079709    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-home\") pod \"e531e587-6554-4305-864e-bfc34a56dd41\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") "
Jul 15 19:54:27.079792 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.079784    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-model-cache\") pod \"e531e587-6554-4305-864e-bfc34a56dd41\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") "
Jul 15 19:54:27.080013 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.079809    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-tmp-dir\") pod \"e531e587-6554-4305-864e-bfc34a56dd41\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") "
Jul 15 19:54:27.080013 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.079864    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9p629\" (UniqueName: \"kubernetes.io/projected/e531e587-6554-4305-864e-bfc34a56dd41-kube-api-access-9p629\") pod \"e531e587-6554-4305-864e-bfc34a56dd41\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") "
Jul 15 19:54:27.080013 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.079892    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-dshm\") pod \"e531e587-6554-4305-864e-bfc34a56dd41\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") "
Jul 15 19:54:27.080013 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.079995    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e531e587-6554-4305-864e-bfc34a56dd41-tls-certs\") pod \"e531e587-6554-4305-864e-bfc34a56dd41\" (UID: \"e531e587-6554-4305-864e-bfc34a56dd41\") "
Jul 15 19:54:27.080013 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.080006    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-home" (OuterVolumeSpecName: "home") pod "e531e587-6554-4305-864e-bfc34a56dd41" (UID: "e531e587-6554-4305-864e-bfc34a56dd41"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:54:27.080278 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.080024    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-model-cache" (OuterVolumeSpecName: "model-cache") pod "e531e587-6554-4305-864e-bfc34a56dd41" (UID: "e531e587-6554-4305-864e-bfc34a56dd41"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:54:27.080278 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.080222    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-model-cache\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:27.080278 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.080235    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-home\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:27.080519 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.080493    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e531e587-6554-4305-864e-bfc34a56dd41" (UID: "e531e587-6554-4305-864e-bfc34a56dd41"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:54:27.082540 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.082507    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-dshm" (OuterVolumeSpecName: "dshm") pod "e531e587-6554-4305-864e-bfc34a56dd41" (UID: "e531e587-6554-4305-864e-bfc34a56dd41"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:54:27.082652 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.082567    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e531e587-6554-4305-864e-bfc34a56dd41-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e531e587-6554-4305-864e-bfc34a56dd41" (UID: "e531e587-6554-4305-864e-bfc34a56dd41"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:54:27.082709 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.082654    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e531e587-6554-4305-864e-bfc34a56dd41-kube-api-access-9p629" (OuterVolumeSpecName: "kube-api-access-9p629") pod "e531e587-6554-4305-864e-bfc34a56dd41" (UID: "e531e587-6554-4305-864e-bfc34a56dd41"). InnerVolumeSpecName "kube-api-access-9p629". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:54:27.167337 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.167307    2560 generic.go:356] "Generic (PLEG): container finished" podID="e531e587-6554-4305-864e-bfc34a56dd41" containerID="8155848d3ed5fe349d88f6e8f429a0b565bf4ac97fc5961e6e9b33d5775813cc" exitCode=0
Jul 15 19:54:27.167584 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.167351    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"
Jul 15 19:54:27.167584 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.167391    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2" event={"ID":"e531e587-6554-4305-864e-bfc34a56dd41","Type":"ContainerDied","Data":"8155848d3ed5fe349d88f6e8f429a0b565bf4ac97fc5961e6e9b33d5775813cc"}
Jul 15 19:54:27.167584 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.167431    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2" event={"ID":"e531e587-6554-4305-864e-bfc34a56dd41","Type":"ContainerDied","Data":"dc0ad525755ee5cccb9a56b8b040afe4f9857adc4444c4bb71c273de6be0bcb1"}
Jul 15 19:54:27.167584 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.167466    2560 scope.go:117] "RemoveContainer" containerID="8155848d3ed5fe349d88f6e8f429a0b565bf4ac97fc5961e6e9b33d5775813cc"
Jul 15 19:54:27.176339 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.176320    2560 scope.go:117] "RemoveContainer" containerID="8155848d3ed5fe349d88f6e8f429a0b565bf4ac97fc5961e6e9b33d5775813cc"
Jul 15 19:54:27.176600 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:54:27.176579    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8155848d3ed5fe349d88f6e8f429a0b565bf4ac97fc5961e6e9b33d5775813cc\": container with ID starting with 8155848d3ed5fe349d88f6e8f429a0b565bf4ac97fc5961e6e9b33d5775813cc not found: ID does not exist" containerID="8155848d3ed5fe349d88f6e8f429a0b565bf4ac97fc5961e6e9b33d5775813cc"
Jul 15 19:54:27.176648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.176609    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8155848d3ed5fe349d88f6e8f429a0b565bf4ac97fc5961e6e9b33d5775813cc"} err="failed to get container status \"8155848d3ed5fe349d88f6e8f429a0b565bf4ac97fc5961e6e9b33d5775813cc\": rpc error: code = NotFound desc = could not find container \"8155848d3ed5fe349d88f6e8f429a0b565bf4ac97fc5961e6e9b33d5775813cc\": container with ID starting with 8155848d3ed5fe349d88f6e8f429a0b565bf4ac97fc5961e6e9b33d5775813cc not found: ID does not exist"
Jul 15 19:54:27.180936 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.180913    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e531e587-6554-4305-864e-bfc34a56dd41-tls-certs\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:27.181047 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.180939    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-tmp-dir\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:27.181047 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.180953    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9p629\" (UniqueName: \"kubernetes.io/projected/e531e587-6554-4305-864e-bfc34a56dd41-kube-api-access-9p629\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:27.181047 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.180968    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e531e587-6554-4305-864e-bfc34a56dd41-dshm\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:27.189037 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.189014    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"]
Jul 15 19:54:27.192296 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.192273    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-da0342/test-model-da0342-kserve-8465ff8df-xddt2"]
Jul 15 19:54:27.674389 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.674353    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc" podUID="ff12b272-fba4-4aae-84fc-66ec2754d1e3" containerName="main" containerID="cri-o://465ee3044429d33e067762811eedb7dfb154d29d2d1d07da6ecc0aa4f2b08795" gracePeriod=45
Jul 15 19:54:27.902788 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.902750    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:27.926355 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.926297    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:54:27.989271 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.989237    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-home\") pod \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") "
Jul 15 19:54:27.989468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.989304    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff12b272-fba4-4aae-84fc-66ec2754d1e3-tls-certs\") pod \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") "
Jul 15 19:54:27.989468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.989338    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-model-cache\") pod \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") "
Jul 15 19:54:27.989468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.989398    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-tmp-dir\") pod \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") "
Jul 15 19:54:27.989468 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.989425    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-dshm\") pod \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") "
Jul 15 19:54:27.989685 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.989509    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z55q\" (UniqueName: \"kubernetes.io/projected/ff12b272-fba4-4aae-84fc-66ec2754d1e3-kube-api-access-6z55q\") pod \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\" (UID: \"ff12b272-fba4-4aae-84fc-66ec2754d1e3\") "
Jul 15 19:54:27.989685 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.989549    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-home" (OuterVolumeSpecName: "home") pod "ff12b272-fba4-4aae-84fc-66ec2754d1e3" (UID: "ff12b272-fba4-4aae-84fc-66ec2754d1e3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:54:27.989766 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.989683    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff12b272-fba4-4aae-84fc-66ec2754d1e3" (UID: "ff12b272-fba4-4aae-84fc-66ec2754d1e3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:54:27.989766 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.989725    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff12b272-fba4-4aae-84fc-66ec2754d1e3" (UID: "ff12b272-fba4-4aae-84fc-66ec2754d1e3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:54:27.989903 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.989879    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-home\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:27.989903 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.989904    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-model-cache\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:27.990088 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.989941    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-tmp-dir\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:27.991620 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.991593    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff12b272-fba4-4aae-84fc-66ec2754d1e3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff12b272-fba4-4aae-84fc-66ec2754d1e3" (UID: "ff12b272-fba4-4aae-84fc-66ec2754d1e3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:54:27.991699 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.991645    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff12b272-fba4-4aae-84fc-66ec2754d1e3-kube-api-access-6z55q" (OuterVolumeSpecName: "kube-api-access-6z55q") pod "ff12b272-fba4-4aae-84fc-66ec2754d1e3" (UID: "ff12b272-fba4-4aae-84fc-66ec2754d1e3"). InnerVolumeSpecName "kube-api-access-6z55q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:54:27.991699 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:27.991647    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-dshm" (OuterVolumeSpecName: "dshm") pod "ff12b272-fba4-4aae-84fc-66ec2754d1e3" (UID: "ff12b272-fba4-4aae-84fc-66ec2754d1e3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:54:28.090927 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.090891    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff12b272-fba4-4aae-84fc-66ec2754d1e3-tls-certs\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:28.090927 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.090923    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff12b272-fba4-4aae-84fc-66ec2754d1e3-dshm\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:28.090927 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.090934    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z55q\" (UniqueName: \"kubernetes.io/projected/ff12b272-fba4-4aae-84fc-66ec2754d1e3-kube-api-access-6z55q\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:28.174889 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.174853    2560 generic.go:356] "Generic (PLEG): container finished" podID="ff12b272-fba4-4aae-84fc-66ec2754d1e3" containerID="465ee3044429d33e067762811eedb7dfb154d29d2d1d07da6ecc0aa4f2b08795" exitCode=0
Jul 15 19:54:28.175049 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.174897    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"
Jul 15 19:54:28.175049 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.174939    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc" event={"ID":"ff12b272-fba4-4aae-84fc-66ec2754d1e3","Type":"ContainerDied","Data":"465ee3044429d33e067762811eedb7dfb154d29d2d1d07da6ecc0aa4f2b08795"}
Jul 15 19:54:28.175049 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.174984    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc" event={"ID":"ff12b272-fba4-4aae-84fc-66ec2754d1e3","Type":"ContainerDied","Data":"68630da94e4ef88cd1ff514dc14e2790c47606e4b92dd1c61021fd36f9bf70ba"}
Jul 15 19:54:28.175049 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.175005    2560 scope.go:117] "RemoveContainer" containerID="465ee3044429d33e067762811eedb7dfb154d29d2d1d07da6ecc0aa4f2b08795"
Jul 15 19:54:28.184129 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.184112    2560 scope.go:117] "RemoveContainer" containerID="465ee3044429d33e067762811eedb7dfb154d29d2d1d07da6ecc0aa4f2b08795"
Jul 15 19:54:28.184385 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:54:28.184366    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"465ee3044429d33e067762811eedb7dfb154d29d2d1d07da6ecc0aa4f2b08795\": container with ID starting with 465ee3044429d33e067762811eedb7dfb154d29d2d1d07da6ecc0aa4f2b08795 not found: ID does not exist" containerID="465ee3044429d33e067762811eedb7dfb154d29d2d1d07da6ecc0aa4f2b08795"
Jul 15 19:54:28.184440 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.184395    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"465ee3044429d33e067762811eedb7dfb154d29d2d1d07da6ecc0aa4f2b08795"} err="failed to get container status \"465ee3044429d33e067762811eedb7dfb154d29d2d1d07da6ecc0aa4f2b08795\": rpc error: code = NotFound desc = could not find container \"465ee3044429d33e067762811eedb7dfb154d29d2d1d07da6ecc0aa4f2b08795\": container with ID starting with 465ee3044429d33e067762811eedb7dfb154d29d2d1d07da6ecc0aa4f2b08795 not found: ID does not exist"
Jul 15 19:54:28.196153 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.196135    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"]
Jul 15 19:54:28.199435 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.199411    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-435e1e/test-model-435e1e-kserve-74cf74cddc-b49cc"]
Jul 15 19:54:28.329613 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.329578    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:28.495705 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.495673    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e531e587-6554-4305-864e-bfc34a56dd41" path="/var/lib/kubelet/pods/e531e587-6554-4305-864e-bfc34a56dd41/volumes"
Jul 15 19:54:28.496036 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:28.496022    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff12b272-fba4-4aae-84fc-66ec2754d1e3" path="/var/lib/kubelet/pods/ff12b272-fba4-4aae-84fc-66ec2754d1e3/volumes"
Jul 15 19:54:34.213900 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.213861    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:34.289134 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.289098    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:34.496941 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.496904    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:34.722716 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.722678    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:34.845770 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.845698    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m"]
Jul 15 19:54:34.846062 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.846049    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e531e587-6554-4305-864e-bfc34a56dd41" containerName="main"
Jul 15 19:54:34.846103 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.846063    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e531e587-6554-4305-864e-bfc34a56dd41" containerName="main"
Jul 15 19:54:34.846103 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.846072    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff12b272-fba4-4aae-84fc-66ec2754d1e3" containerName="main"
Jul 15 19:54:34.846103 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.846077    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff12b272-fba4-4aae-84fc-66ec2754d1e3" containerName="main"
Jul 15 19:54:34.846103 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.846093    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baecea79-9943-4766-9212-aa46ff27f4d9" containerName="revoke-keys"
Jul 15 19:54:34.846103 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.846098    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="baecea79-9943-4766-9212-aa46ff27f4d9" containerName="revoke-keys"
Jul 15 19:54:34.846255 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.846162    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="baecea79-9943-4766-9212-aa46ff27f4d9" containerName="revoke-keys"
Jul 15 19:54:34.846255 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.846171    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff12b272-fba4-4aae-84fc-66ec2754d1e3" containerName="main"
Jul 15 19:54:34.846255 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.846178    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e531e587-6554-4305-864e-bfc34a56dd41" containerName="main"
Jul 15 19:54:34.848803 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.848786    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m"
Jul 15 19:54:34.851273 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.851252    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 19:54:34.851413 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.851324    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bk6kr\""
Jul 15 19:54:34.856272 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.856254    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m"]
Jul 15 19:54:34.948156 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:34.948131    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bf4b6bf-5c88-464a-af9d-7a64649889ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m\" (UID: \"5bf4b6bf-5c88-464a-af9d-7a64649889ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m"
Jul 15 19:54:35.049236 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:35.049208    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bf4b6bf-5c88-464a-af9d-7a64649889ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m\" (UID: \"5bf4b6bf-5c88-464a-af9d-7a64649889ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m"
Jul 15 19:54:35.049767 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:35.049750    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bf4b6bf-5c88-464a-af9d-7a64649889ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m\" (UID: \"5bf4b6bf-5c88-464a-af9d-7a64649889ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m"
Jul 15 19:54:35.160604 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:35.160519    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m"
Jul 15 19:54:35.293865 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:35.293840    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m"]
Jul 15 19:54:35.295808 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:54:35.295781    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf4b6bf_5c88_464a_af9d_7a64649889ec.slice/crio-bf254eacbb73c95d3b30871caa61726b39630e0c17f252557d0485e83ca3d807 WatchSource:0}: Error finding container bf254eacbb73c95d3b30871caa61726b39630e0c17f252557d0485e83ca3d807: Status 404 returned error can't find the container with id bf254eacbb73c95d3b30871caa61726b39630e0c17f252557d0485e83ca3d807
Jul 15 19:54:35.536595 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:35.536561    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:35.608477 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:35.608434    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:36.216944 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:36.216900    2560 generic.go:356] "Generic (PLEG): container finished" podID="5bf4b6bf-5c88-464a-af9d-7a64649889ec" containerID="a19940bc7b086c881bd17f951aab4efb30bdbd49b89a26dfcce5e76e118c7436" exitCode=0
Jul 15 19:54:36.217107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:36.216987    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m" event={"ID":"5bf4b6bf-5c88-464a-af9d-7a64649889ec","Type":"ContainerDied","Data":"a19940bc7b086c881bd17f951aab4efb30bdbd49b89a26dfcce5e76e118c7436"}
Jul 15 19:54:36.217107 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:36.217036    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m" event={"ID":"5bf4b6bf-5c88-464a-af9d-7a64649889ec","Type":"ContainerStarted","Data":"bf254eacbb73c95d3b30871caa61726b39630e0c17f252557d0485e83ca3d807"}
Jul 15 19:54:37.349331 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:37.349308    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m"
Jul 15 19:54:37.472463 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:37.472388    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bf4b6bf-5c88-464a-af9d-7a64649889ec-maas-api-service-ca\") pod \"5bf4b6bf-5c88-464a-af9d-7a64649889ec\" (UID: \"5bf4b6bf-5c88-464a-af9d-7a64649889ec\") "
Jul 15 19:54:37.472718 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:37.472699    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bf4b6bf-5c88-464a-af9d-7a64649889ec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5bf4b6bf-5c88-464a-af9d-7a64649889ec" (UID: "5bf4b6bf-5c88-464a-af9d-7a64649889ec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:54:37.573414 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:37.573387    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bf4b6bf-5c88-464a-af9d-7a64649889ec-maas-api-service-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:54:38.227716 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:38.227680    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m" event={"ID":"5bf4b6bf-5c88-464a-af9d-7a64649889ec","Type":"ContainerDied","Data":"bf254eacbb73c95d3b30871caa61726b39630e0c17f252557d0485e83ca3d807"}
Jul 15 19:54:38.227716 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:38.227718    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m"
Jul 15 19:54:38.227923 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:38.227754    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf254eacbb73c95d3b30871caa61726b39630e0c17f252557d0485e83ca3d807"
Jul 15 19:54:45.332012 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:45.331976    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:49.925648 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:49.925613    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:50.344202 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:50.344167    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:56.011752 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:56.011716    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:56.112703 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:56.112668    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:56.292098 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:56.292002    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:56.531013 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:56.530976    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:56.916376 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:56.916343    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:57.236343 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:57.236312    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:54:57.460105 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:54:57.460073    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:00.038477 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:00.038424    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:00.116981 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:00.116942    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:00.190258 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:00.190222    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:00.319873 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:00.319794    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:00.793398 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:00.793365    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:00.895978 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:00.895945    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:04.556232 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:04.556197    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:05.033487 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:05.033444    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:05.328594 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:05.328512    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:05.903898 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:05.903865    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:08.076297 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:08.076260    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:08.159844 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:08.159804    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:08.223080 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:08.223052    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:08.323434 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:08.323403    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:08.656186 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:08.656150    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:08.868211 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:08.868185    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:12.588632 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:12.588599    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:20.727080 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:20.727045    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7"]
Jul 15 19:55:20.727467 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:20.727439    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bf4b6bf-5c88-464a-af9d-7a64649889ec" containerName="revoke-keys"
Jul 15 19:55:20.727467 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:20.727464    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bf4b6bf-5c88-464a-af9d-7a64649889ec" containerName="revoke-keys"
Jul 15 19:55:20.727545 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:20.727517    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bf4b6bf-5c88-464a-af9d-7a64649889ec" containerName="revoke-keys"
Jul 15 19:55:20.730694 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:20.730677    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7"
Jul 15 19:55:20.733248 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:20.733221    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 19:55:20.733248 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:20.733222    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bk6kr\""
Jul 15 19:55:20.737900 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:20.737844    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7"]
Jul 15 19:55:20.863900 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:20.863860    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17990a8-c2a4-453a-9b95-14dfebe13c89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7\" (UID: \"d17990a8-c2a4-453a-9b95-14dfebe13c89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7"
Jul 15 19:55:20.965317 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:20.965278    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17990a8-c2a4-453a-9b95-14dfebe13c89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7\" (UID: \"d17990a8-c2a4-453a-9b95-14dfebe13c89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7"
Jul 15 19:55:20.965984 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:20.965964    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17990a8-c2a4-453a-9b95-14dfebe13c89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7\" (UID: \"d17990a8-c2a4-453a-9b95-14dfebe13c89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7"
Jul 15 19:55:21.042417 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:21.042367    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7"
Jul 15 19:55:21.166625 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:21.166596    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7"]
Jul 15 19:55:21.167964 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:55:21.167938    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd17990a8_c2a4_453a_9b95_14dfebe13c89.slice/crio-985a59c8ebc79c11722824af723ab36f816351ca668f295f51bb40a8329b9b3a WatchSource:0}: Error finding container 985a59c8ebc79c11722824af723ab36f816351ca668f295f51bb40a8329b9b3a: Status 404 returned error can't find the container with id 985a59c8ebc79c11722824af723ab36f816351ca668f295f51bb40a8329b9b3a
Jul 15 19:55:21.347830 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:21.347749    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:21.435322 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:21.435288    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7" event={"ID":"d17990a8-c2a4-453a-9b95-14dfebe13c89","Type":"ContainerStarted","Data":"91dcf5e8aa1555142bcc06ca677678d1a58b9484d928b33180b23620e73f8e08"}
Jul 15 19:55:21.435499 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:21.435373    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7" event={"ID":"d17990a8-c2a4-453a-9b95-14dfebe13c89","Type":"ContainerStarted","Data":"985a59c8ebc79c11722824af723ab36f816351ca668f295f51bb40a8329b9b3a"}
Jul 15 19:55:21.447545 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:21.447519    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:21.453168 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:21.453129    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7" podStartSLOduration=1.453116135 podStartE2EDuration="1.453116135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:21.452086971 +0000 UTC m=+2393.521946369" watchObservedRunningTime="2026-07-15 19:55:21.453116135 +0000 UTC m=+2393.522975531"
Jul 15 19:55:22.441624 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:22.441588    2560 generic.go:356] "Generic (PLEG): container finished" podID="d17990a8-c2a4-453a-9b95-14dfebe13c89" containerID="91dcf5e8aa1555142bcc06ca677678d1a58b9484d928b33180b23620e73f8e08" exitCode=0
Jul 15 19:55:22.442011 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:22.441673    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7" event={"ID":"d17990a8-c2a4-453a-9b95-14dfebe13c89","Type":"ContainerDied","Data":"91dcf5e8aa1555142bcc06ca677678d1a58b9484d928b33180b23620e73f8e08"}
Jul 15 19:55:23.576960 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:23.576933    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7"
Jul 15 19:55:23.694015 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:23.693980    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17990a8-c2a4-453a-9b95-14dfebe13c89-maas-api-service-ca\") pod \"d17990a8-c2a4-453a-9b95-14dfebe13c89\" (UID: \"d17990a8-c2a4-453a-9b95-14dfebe13c89\") "
Jul 15 19:55:23.694388 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:23.694367    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d17990a8-c2a4-453a-9b95-14dfebe13c89-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d17990a8-c2a4-453a-9b95-14dfebe13c89" (UID: "d17990a8-c2a4-453a-9b95-14dfebe13c89"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:55:23.795426 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:23.795394    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17990a8-c2a4-453a-9b95-14dfebe13c89-maas-api-service-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:55:24.453125 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:24.453093    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7" event={"ID":"d17990a8-c2a4-453a-9b95-14dfebe13c89","Type":"ContainerDied","Data":"985a59c8ebc79c11722824af723ab36f816351ca668f295f51bb40a8329b9b3a"}
Jul 15 19:55:24.453297 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:24.453132    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7"
Jul 15 19:55:24.453297 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:24.453140    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="985a59c8ebc79c11722824af723ab36f816351ca668f295f51bb40a8329b9b3a"
Jul 15 19:55:27.124873 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:27.124828    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:27.224221 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:27.224186    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:28.556478 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:28.556438    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:55:28.559469 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:28.559435    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 19:55:29.467512 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:29.467477    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:29.568886 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:29.568847    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:31.386222 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:31.386181    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:31.493076 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:31.493038    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:36.648364 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:36.648326    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:40.820512 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:40.820445    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:41.194882 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:41.194778    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:46.982059 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:46.982013    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:47.082699 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.082665    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:47.239835 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.239803    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:47.456532 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.456493    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:47.649167 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.649075    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll"]
Jul 15 19:55:47.649572 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.649555    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d17990a8-c2a4-453a-9b95-14dfebe13c89" containerName="revoke-keys"
Jul 15 19:55:47.649678 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.649574    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d17990a8-c2a4-453a-9b95-14dfebe13c89" containerName="revoke-keys"
Jul 15 19:55:47.649678 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.649654    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d17990a8-c2a4-453a-9b95-14dfebe13c89" containerName="revoke-keys"
Jul 15 19:55:47.652790 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.652769    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll"
Jul 15 19:55:47.656783 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.656758    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bk6kr\""
Jul 15 19:55:47.657897 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.657875    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 19:55:47.667362 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.667341    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll"]
Jul 15 19:55:47.719072 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.719037    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24d18cd0-cfb9-4374-95e0-7159f82e3e2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll\" (UID: \"24d18cd0-cfb9-4374-95e0-7159f82e3e2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll"
Jul 15 19:55:47.819625 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.819587    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24d18cd0-cfb9-4374-95e0-7159f82e3e2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll\" (UID: \"24d18cd0-cfb9-4374-95e0-7159f82e3e2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll"
Jul 15 19:55:47.820205 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.820185    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24d18cd0-cfb9-4374-95e0-7159f82e3e2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll\" (UID: \"24d18cd0-cfb9-4374-95e0-7159f82e3e2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll"
Jul 15 19:55:47.962309 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:47.962225    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll"
Jul 15 19:55:48.096091 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:48.095960    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll"]
Jul 15 19:55:48.098996 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:55:48.098957    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d18cd0_cfb9_4374_95e0_7159f82e3e2a.slice/crio-4c0a83e01a79806787858a79f98ef796d5b9dfe69712206893445679a36ee31b WatchSource:0}: Error finding container 4c0a83e01a79806787858a79f98ef796d5b9dfe69712206893445679a36ee31b: Status 404 returned error can't find the container with id 4c0a83e01a79806787858a79f98ef796d5b9dfe69712206893445679a36ee31b
Jul 15 19:55:48.141569 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:48.141542    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:48.243235 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:48.243201    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:55:48.573068 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:48.572964    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll" event={"ID":"24d18cd0-cfb9-4374-95e0-7159f82e3e2a","Type":"ContainerStarted","Data":"2699c59b445954eeb04431df34998fd4c2719d7f43beb5c9fa8ee42a981dcb60"}
Jul 15 19:55:48.573068 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:48.573020    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll" event={"ID":"24d18cd0-cfb9-4374-95e0-7159f82e3e2a","Type":"ContainerStarted","Data":"4c0a83e01a79806787858a79f98ef796d5b9dfe69712206893445679a36ee31b"}
Jul 15 19:55:48.590721 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:48.590669    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll" podStartSLOduration=1.590652331 podStartE2EDuration="1.590652331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:55:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:48.589385734 +0000 UTC m=+2420.659245133" watchObservedRunningTime="2026-07-15 19:55:48.590652331 +0000 UTC m=+2420.660511728"
Jul 15 19:55:49.580106 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:49.580069    2560 generic.go:356] "Generic (PLEG): container finished" podID="24d18cd0-cfb9-4374-95e0-7159f82e3e2a" containerID="2699c59b445954eeb04431df34998fd4c2719d7f43beb5c9fa8ee42a981dcb60" exitCode=0
Jul 15 19:55:49.580531 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:49.580114    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll" event={"ID":"24d18cd0-cfb9-4374-95e0-7159f82e3e2a","Type":"ContainerDied","Data":"2699c59b445954eeb04431df34998fd4c2719d7f43beb5c9fa8ee42a981dcb60"}
Jul 15 19:55:50.731132 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:50.731105    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll"
Jul 15 19:55:50.844503 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:50.844372    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24d18cd0-cfb9-4374-95e0-7159f82e3e2a-maas-api-service-ca\") pod \"24d18cd0-cfb9-4374-95e0-7159f82e3e2a\" (UID: \"24d18cd0-cfb9-4374-95e0-7159f82e3e2a\") "
Jul 15 19:55:50.844810 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:50.844789    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24d18cd0-cfb9-4374-95e0-7159f82e3e2a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "24d18cd0-cfb9-4374-95e0-7159f82e3e2a" (UID: "24d18cd0-cfb9-4374-95e0-7159f82e3e2a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 19:55:50.945878 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:50.945840    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24d18cd0-cfb9-4374-95e0-7159f82e3e2a-maas-api-service-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:55:51.590493 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:51.590436    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll"
Jul 15 19:55:51.590493 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:51.590474    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll" event={"ID":"24d18cd0-cfb9-4374-95e0-7159f82e3e2a","Type":"ContainerDied","Data":"4c0a83e01a79806787858a79f98ef796d5b9dfe69712206893445679a36ee31b"}
Jul 15 19:55:51.590715 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:55:51.590524    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c0a83e01a79806787858a79f98ef796d5b9dfe69712206893445679a36ee31b"
Jul 15 19:56:03.221061 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:03.221029    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:07.699123 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:07.699083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:08.074081 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:08.074041    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:13.978796 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:13.978754    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:14.078611 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:14.078573    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:14.248768 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:14.248736    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:14.558082 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:14.557990    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:14.876689 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:14.876598    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:15.204358 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:15.204269    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:15.854563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:15.854523    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:18.553273 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:18.553229    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:18.636644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:18.636603    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:18.730537 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:18.730498    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:18.879557 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:18.879431    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:19.200770 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:19.200673    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:19.300325 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:19.300288    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:23.009062 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:23.009028    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:23.280384 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:23.280298    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:23.604273 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:23.604184    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:23.707673 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:23.707641    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:26.471245 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:26.471212    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:26.553991 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:26.553954    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:26.625740 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:26.625697    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:26.955994 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:26.955953    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:27.118894 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:27.118857    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:27.217793 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:27.217696    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:30.927629 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:30.927594    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:33.911409 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:33.911365    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:33.963859 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:33.963817    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:39.947541 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:39.947501    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:40.045147 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:40.045109    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:42.467519 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:42.467479    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:43.037863 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.037832    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"]
Jul 15 19:56:43.038211 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.038199    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24d18cd0-cfb9-4374-95e0-7159f82e3e2a" containerName="revoke-keys"
Jul 15 19:56:43.038262 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.038212    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="24d18cd0-cfb9-4374-95e0-7159f82e3e2a" containerName="revoke-keys"
Jul 15 19:56:43.038297 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.038281    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="24d18cd0-cfb9-4374-95e0-7159f82e3e2a" containerName="revoke-keys"
Jul 15 19:56:43.041127 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.041108    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.043850 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.043826    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e8718e\"/\"kube-root-ca.crt\""
Jul 15 19:56:43.045001 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.044984    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e8718e\"/\"openshift-service-ca.crt\""
Jul 15 19:56:43.045098 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.045014    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e8718e\"/\"auth-test-model-e8718e-kserve-self-signed-certs\""
Jul 15 19:56:43.045098 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.045049    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e8718e\"/\"default-dockercfg-5ggtc\""
Jul 15 19:56:43.050739 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.050717    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"]
Jul 15 19:56:43.127305 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.127261    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-model-cache\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.127305 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.127307    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxtwt\" (UniqueName: \"kubernetes.io/projected/ce416cd5-54d1-438a-94b7-1cb06d859b6d-kube-api-access-sxtwt\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.127563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.127331    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-tmp-dir\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.127563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.127427    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-home\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.127563 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.127515    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-dshm\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.127666 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.127576    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce416cd5-54d1-438a-94b7-1cb06d859b6d-tls-certs\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.183507 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.183471    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:43.228303 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.228264    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce416cd5-54d1-438a-94b7-1cb06d859b6d-tls-certs\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.228512 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.228341    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-model-cache\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.228512 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.228371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxtwt\" (UniqueName: \"kubernetes.io/projected/ce416cd5-54d1-438a-94b7-1cb06d859b6d-kube-api-access-sxtwt\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.228512 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.228404    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-tmp-dir\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.228512 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.228480    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-home\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.228726 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.228521    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-dshm\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.228925 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.228894    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-tmp-dir\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.229005 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.228952    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-model-cache\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.229005 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.228988    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-home\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.230712 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.230687    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-dshm\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.230962 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.230946    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce416cd5-54d1-438a-94b7-1cb06d859b6d-tls-certs\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.236812 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.236788    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxtwt\" (UniqueName: \"kubernetes.io/projected/ce416cd5-54d1-438a-94b7-1cb06d859b6d-kube-api-access-sxtwt\") pod \"auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") " pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.367321 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.367222    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:43.509698 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.509673    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"]
Jul 15 19:56:43.511744 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:56:43.511710    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce416cd5_54d1_438a_94b7_1cb06d859b6d.slice/crio-7a6d1882c1fafcfd28d784a04e3458e23a8e15c29b5c08ea91e9045d5cc67038 WatchSource:0}: Error finding container 7a6d1882c1fafcfd28d784a04e3458e23a8e15c29b5c08ea91e9045d5cc67038: Status 404 returned error can't find the container with id 7a6d1882c1fafcfd28d784a04e3458e23a8e15c29b5c08ea91e9045d5cc67038
Jul 15 19:56:43.846116 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.846081    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp" event={"ID":"ce416cd5-54d1-438a-94b7-1cb06d859b6d","Type":"ContainerStarted","Data":"b09a6a6c7409526fd5187d319f073d2006d428dc9fcea74815cf1d74bc746131"}
Jul 15 19:56:43.846116 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.846122    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp" event={"ID":"ce416cd5-54d1-438a-94b7-1cb06d859b6d","Type":"ContainerStarted","Data":"7a6d1882c1fafcfd28d784a04e3458e23a8e15c29b5c08ea91e9045d5cc67038"}
Jul 15 19:56:43.866504 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:43.866420    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp" podStartSLOduration=0.866402749 podStartE2EDuration="866.402749ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:56:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:43.864659726 +0000 UTC m=+2475.934519125" watchObservedRunningTime="2026-07-15 19:56:43.866402749 +0000 UTC m=+2475.936262147"
Jul 15 19:56:47.849620 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:47.849581    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:47.950745 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:47.950707    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:48.281566 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:48.281519    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:56:53.367827 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:53.367796    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:53.368200 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:53.367845    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:53.380990 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:53.380967    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:56:53.905978 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:56:53.905944    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:57:00.742400 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.742367    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"]
Jul 15 19:57:00.745790 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.745773    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.749415 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.749389    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-613f78\"/\"default-dockercfg-7w8dd\""
Jul 15 19:57:00.749551 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.749436    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-613f78\"/\"auth-test-model-613f78-kserve-self-signed-certs\""
Jul 15 19:57:00.749610 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.749566    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-613f78\"/\"kube-root-ca.crt\""
Jul 15 19:57:00.749694 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.749680    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-613f78\"/\"openshift-service-ca.crt\""
Jul 15 19:57:00.755198 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.755170    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"]
Jul 15 19:57:00.773494 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.773462    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:00.793940 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.793904    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-tmp-dir\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.794090 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.793965    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-home\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.794090 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.794004    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnqtl\" (UniqueName: \"kubernetes.io/projected/b2036e60-1484-490d-884d-1fd3a29f152a-kube-api-access-fnqtl\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.794090 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.794065    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2036e60-1484-490d-884d-1fd3a29f152a-tls-certs\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.794266 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.794133    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-model-cache\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.794266 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.794183    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-dshm\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.895564 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.895518    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-tmp-dir\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.895778 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.895576    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-home\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.895778 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.895614    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnqtl\" (UniqueName: \"kubernetes.io/projected/b2036e60-1484-490d-884d-1fd3a29f152a-kube-api-access-fnqtl\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.895778 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.895642    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2036e60-1484-490d-884d-1fd3a29f152a-tls-certs\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.895778 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.895691    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-model-cache\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.896002 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.895874    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-dshm\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.896002 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.895977    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-tmp-dir\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.896120 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.896029    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-home\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.896120 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.896097    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-model-cache\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.898139 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.898112    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-dshm\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.898585 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.898564    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2036e60-1484-490d-884d-1fd3a29f152a-tls-certs\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:00.903544 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:00.903521    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnqtl\" (UniqueName: \"kubernetes.io/projected/b2036e60-1484-490d-884d-1fd3a29f152a-kube-api-access-fnqtl\") pod \"auth-test-model-613f78-kserve-7fc7bf5df5-q96mq\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") " pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:01.057442 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:01.057346    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:01.194645 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:01.194618    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"]
Jul 15 19:57:01.197317 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:57:01.197286    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2036e60_1484_490d_884d_1fd3a29f152a.slice/crio-f9a3bd1c8d1d8630aacbbf135c1a42e4a9fb991f64e586db29c5b516f40e1007 WatchSource:0}: Error finding container f9a3bd1c8d1d8630aacbbf135c1a42e4a9fb991f64e586db29c5b516f40e1007: Status 404 returned error can't find the container with id f9a3bd1c8d1d8630aacbbf135c1a42e4a9fb991f64e586db29c5b516f40e1007
Jul 15 19:57:01.942023 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:01.941986    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq" event={"ID":"b2036e60-1484-490d-884d-1fd3a29f152a","Type":"ContainerStarted","Data":"5f05f9b5825d950a83691d69841f348589e0e6307dfa54b67c2868f64ef17301"}
Jul 15 19:57:01.942023 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:01.942028    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq" event={"ID":"b2036e60-1484-490d-884d-1fd3a29f152a","Type":"ContainerStarted","Data":"f9a3bd1c8d1d8630aacbbf135c1a42e4a9fb991f64e586db29c5b516f40e1007"}
Jul 15 19:57:01.958254 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:01.958186    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq" podStartSLOduration=1.958167446 podStartE2EDuration="1.958167446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:57:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:57:01.957539092 +0000 UTC m=+2494.027398491" watchObservedRunningTime="2026-07-15 19:57:01.958167446 +0000 UTC m=+2494.028026845"
Jul 15 19:57:11.057649 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:11.057610    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:11.058159 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:11.057670    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:11.070219 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:11.070193    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:12.006914 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:12.006886    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:57:21.235709 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:21.235625    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:57:21.339676 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:57:21.339645    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:08.719096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:08.719019    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"]
Jul 15 19:59:08.765050 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:08.765018    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:09.733524 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:09.733492    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"]
Jul 15 19:59:09.746916 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:09.746887    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:13.359035 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:13.359002    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:23.738632 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:23.738593    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp" podUID="ce416cd5-54d1-438a-94b7-1cb06d859b6d" containerName="main" containerID="cri-o://b09a6a6c7409526fd5187d319f073d2006d428dc9fcea74815cf1d74bc746131" gracePeriod=45
Jul 15 19:59:23.895041 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:23.894996    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp" podUID="ce416cd5-54d1-438a-94b7-1cb06d859b6d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": dial tcp 10.133.0.49:8000: connect: connection refused"
Jul 15 19:59:23.981179 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:23.981152    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:59:24.168226 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.168137    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-model-cache\") pod \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") "
Jul 15 19:59:24.168226 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.168178    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce416cd5-54d1-438a-94b7-1cb06d859b6d-tls-certs\") pod \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") "
Jul 15 19:59:24.168226 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.168198    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxtwt\" (UniqueName: \"kubernetes.io/projected/ce416cd5-54d1-438a-94b7-1cb06d859b6d-kube-api-access-sxtwt\") pod \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") "
Jul 15 19:59:24.168556 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.168233    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-dshm\") pod \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") "
Jul 15 19:59:24.168556 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.168292    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-home\") pod \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") "
Jul 15 19:59:24.168556 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.168340    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-tmp-dir\") pod \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\" (UID: \"ce416cd5-54d1-438a-94b7-1cb06d859b6d\") "
Jul 15 19:59:24.168556 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.168434    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce416cd5-54d1-438a-94b7-1cb06d859b6d" (UID: "ce416cd5-54d1-438a-94b7-1cb06d859b6d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:59:24.168783 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.168582    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-home" (OuterVolumeSpecName: "home") pod "ce416cd5-54d1-438a-94b7-1cb06d859b6d" (UID: "ce416cd5-54d1-438a-94b7-1cb06d859b6d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:59:24.168783 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.168665    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce416cd5-54d1-438a-94b7-1cb06d859b6d" (UID: "ce416cd5-54d1-438a-94b7-1cb06d859b6d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:59:24.168783 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.168770    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-model-cache\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:59:24.168947 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.168786    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-home\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:59:24.168947 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.168800    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-tmp-dir\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:59:24.170508 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.170481    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-dshm" (OuterVolumeSpecName: "dshm") pod "ce416cd5-54d1-438a-94b7-1cb06d859b6d" (UID: "ce416cd5-54d1-438a-94b7-1cb06d859b6d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:59:24.170610 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.170511    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce416cd5-54d1-438a-94b7-1cb06d859b6d-kube-api-access-sxtwt" (OuterVolumeSpecName: "kube-api-access-sxtwt") pod "ce416cd5-54d1-438a-94b7-1cb06d859b6d" (UID: "ce416cd5-54d1-438a-94b7-1cb06d859b6d"). InnerVolumeSpecName "kube-api-access-sxtwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:59:24.170610 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.170522    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce416cd5-54d1-438a-94b7-1cb06d859b6d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce416cd5-54d1-438a-94b7-1cb06d859b6d" (UID: "ce416cd5-54d1-438a-94b7-1cb06d859b6d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:59:24.269931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.269889    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce416cd5-54d1-438a-94b7-1cb06d859b6d-dshm\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:59:24.269931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.269923    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce416cd5-54d1-438a-94b7-1cb06d859b6d-tls-certs\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:59:24.269931 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.269934    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxtwt\" (UniqueName: \"kubernetes.io/projected/ce416cd5-54d1-438a-94b7-1cb06d859b6d-kube-api-access-sxtwt\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:59:24.665560 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.665522    2560 generic.go:356] "Generic (PLEG): container finished" podID="ce416cd5-54d1-438a-94b7-1cb06d859b6d" containerID="b09a6a6c7409526fd5187d319f073d2006d428dc9fcea74815cf1d74bc746131" exitCode=0
Jul 15 19:59:24.665560 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.665562    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"
Jul 15 19:59:24.665802 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.665605    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp" event={"ID":"ce416cd5-54d1-438a-94b7-1cb06d859b6d","Type":"ContainerDied","Data":"b09a6a6c7409526fd5187d319f073d2006d428dc9fcea74815cf1d74bc746131"}
Jul 15 19:59:24.665802 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.665648    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp" event={"ID":"ce416cd5-54d1-438a-94b7-1cb06d859b6d","Type":"ContainerDied","Data":"7a6d1882c1fafcfd28d784a04e3458e23a8e15c29b5c08ea91e9045d5cc67038"}
Jul 15 19:59:24.665802 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.665667    2560 scope.go:117] "RemoveContainer" containerID="b09a6a6c7409526fd5187d319f073d2006d428dc9fcea74815cf1d74bc746131"
Jul 15 19:59:24.674233 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.674206    2560 scope.go:117] "RemoveContainer" containerID="b09a6a6c7409526fd5187d319f073d2006d428dc9fcea74815cf1d74bc746131"
Jul 15 19:59:24.674504 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:59:24.674485    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b09a6a6c7409526fd5187d319f073d2006d428dc9fcea74815cf1d74bc746131\": container with ID starting with b09a6a6c7409526fd5187d319f073d2006d428dc9fcea74815cf1d74bc746131 not found: ID does not exist" containerID="b09a6a6c7409526fd5187d319f073d2006d428dc9fcea74815cf1d74bc746131"
Jul 15 19:59:24.674560 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.674515    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b09a6a6c7409526fd5187d319f073d2006d428dc9fcea74815cf1d74bc746131"} err="failed to get container status \"b09a6a6c7409526fd5187d319f073d2006d428dc9fcea74815cf1d74bc746131\": rpc error: code = NotFound desc = could not find container \"b09a6a6c7409526fd5187d319f073d2006d428dc9fcea74815cf1d74bc746131\": container with ID starting with b09a6a6c7409526fd5187d319f073d2006d428dc9fcea74815cf1d74bc746131 not found: ID does not exist"
Jul 15 19:59:24.683745 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.683719    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"]
Jul 15 19:59:24.686570 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.686541    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/auth-test-model-e8718e-kserve-59c8bdb7f-5vhbp"]
Jul 15 19:59:24.755178 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:24.755096    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq" podUID="b2036e60-1484-490d-884d-1fd3a29f152a" containerName="main" containerID="cri-o://5f05f9b5825d950a83691d69841f348589e0e6307dfa54b67c2868f64ef17301" gracePeriod=45
Jul 15 19:59:25.000146 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.000120    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:59:25.178178 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.178084    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-dshm\") pod \"b2036e60-1484-490d-884d-1fd3a29f152a\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") "
Jul 15 19:59:25.178178 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.178149    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-tmp-dir\") pod \"b2036e60-1484-490d-884d-1fd3a29f152a\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") "
Jul 15 19:59:25.178421 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.178178    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnqtl\" (UniqueName: \"kubernetes.io/projected/b2036e60-1484-490d-884d-1fd3a29f152a-kube-api-access-fnqtl\") pod \"b2036e60-1484-490d-884d-1fd3a29f152a\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") "
Jul 15 19:59:25.178421 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.178225    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-home\") pod \"b2036e60-1484-490d-884d-1fd3a29f152a\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") "
Jul 15 19:59:25.178421 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.178302    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2036e60-1484-490d-884d-1fd3a29f152a-tls-certs\") pod \"b2036e60-1484-490d-884d-1fd3a29f152a\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") "
Jul 15 19:59:25.178421 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.178344    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-model-cache\") pod \"b2036e60-1484-490d-884d-1fd3a29f152a\" (UID: \"b2036e60-1484-490d-884d-1fd3a29f152a\") "
Jul 15 19:59:25.178679 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.178512    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b2036e60-1484-490d-884d-1fd3a29f152a" (UID: "b2036e60-1484-490d-884d-1fd3a29f152a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:59:25.178740 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.178664    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-home" (OuterVolumeSpecName: "home") pod "b2036e60-1484-490d-884d-1fd3a29f152a" (UID: "b2036e60-1484-490d-884d-1fd3a29f152a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:59:25.178791 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.178731    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-model-cache" (OuterVolumeSpecName: "model-cache") pod "b2036e60-1484-490d-884d-1fd3a29f152a" (UID: "b2036e60-1484-490d-884d-1fd3a29f152a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:59:25.178791 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.178767    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-home\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:59:25.178791 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.178786    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-tmp-dir\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:59:25.180882 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.180853    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2036e60-1484-490d-884d-1fd3a29f152a-kube-api-access-fnqtl" (OuterVolumeSpecName: "kube-api-access-fnqtl") pod "b2036e60-1484-490d-884d-1fd3a29f152a" (UID: "b2036e60-1484-490d-884d-1fd3a29f152a"). InnerVolumeSpecName "kube-api-access-fnqtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 19:59:25.181001 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.180902    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-dshm" (OuterVolumeSpecName: "dshm") pod "b2036e60-1484-490d-884d-1fd3a29f152a" (UID: "b2036e60-1484-490d-884d-1fd3a29f152a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 19:59:25.181001 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.180954    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2036e60-1484-490d-884d-1fd3a29f152a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b2036e60-1484-490d-884d-1fd3a29f152a" (UID: "b2036e60-1484-490d-884d-1fd3a29f152a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 19:59:25.280096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.280064    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2036e60-1484-490d-884d-1fd3a29f152a-tls-certs\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:59:25.280096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.280092    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-model-cache\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:59:25.280096 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.280100    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2036e60-1484-490d-884d-1fd3a29f152a-dshm\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:59:25.280312 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.280109    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnqtl\" (UniqueName: \"kubernetes.io/projected/b2036e60-1484-490d-884d-1fd3a29f152a-kube-api-access-fnqtl\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 19:59:25.672303 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.672270    2560 generic.go:356] "Generic (PLEG): container finished" podID="b2036e60-1484-490d-884d-1fd3a29f152a" containerID="5f05f9b5825d950a83691d69841f348589e0e6307dfa54b67c2868f64ef17301" exitCode=0
Jul 15 19:59:25.672517 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.672310    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq" event={"ID":"b2036e60-1484-490d-884d-1fd3a29f152a","Type":"ContainerDied","Data":"5f05f9b5825d950a83691d69841f348589e0e6307dfa54b67c2868f64ef17301"}
Jul 15 19:59:25.672517 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.672354    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq" event={"ID":"b2036e60-1484-490d-884d-1fd3a29f152a","Type":"ContainerDied","Data":"f9a3bd1c8d1d8630aacbbf135c1a42e4a9fb991f64e586db29c5b516f40e1007"}
Jul 15 19:59:25.672517 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.672370    2560 scope.go:117] "RemoveContainer" containerID="5f05f9b5825d950a83691d69841f348589e0e6307dfa54b67c2868f64ef17301"
Jul 15 19:59:25.672517 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.672319    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"
Jul 15 19:59:25.682238 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.682218    2560 scope.go:117] "RemoveContainer" containerID="5f05f9b5825d950a83691d69841f348589e0e6307dfa54b67c2868f64ef17301"
Jul 15 19:59:25.682526 ip-10-0-143-114 kubenswrapper[2560]: E0715 19:59:25.682504    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f05f9b5825d950a83691d69841f348589e0e6307dfa54b67c2868f64ef17301\": container with ID starting with 5f05f9b5825d950a83691d69841f348589e0e6307dfa54b67c2868f64ef17301 not found: ID does not exist" containerID="5f05f9b5825d950a83691d69841f348589e0e6307dfa54b67c2868f64ef17301"
Jul 15 19:59:25.682644 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.682538    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f05f9b5825d950a83691d69841f348589e0e6307dfa54b67c2868f64ef17301"} err="failed to get container status \"5f05f9b5825d950a83691d69841f348589e0e6307dfa54b67c2868f64ef17301\": rpc error: code = NotFound desc = could not find container \"5f05f9b5825d950a83691d69841f348589e0e6307dfa54b67c2868f64ef17301\": container with ID starting with 5f05f9b5825d950a83691d69841f348589e0e6307dfa54b67c2868f64ef17301 not found: ID does not exist"
Jul 15 19:59:25.694693 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.694665    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"]
Jul 15 19:59:25.700881 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:25.700856    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/auth-test-model-613f78-kserve-7fc7bf5df5-q96mq"]
Jul 15 19:59:26.494987 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:26.494946    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2036e60-1484-490d-884d-1fd3a29f152a" path="/var/lib/kubelet/pods/b2036e60-1484-490d-884d-1fd3a29f152a/volumes"
Jul 15 19:59:26.495561 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:26.495539    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce416cd5-54d1-438a-94b7-1cb06d859b6d" path="/var/lib/kubelet/pods/ce416cd5-54d1-438a-94b7-1cb06d859b6d/volumes"
Jul 15 19:59:43.841485 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.841437    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"]
Jul 15 19:59:43.842217 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.842199    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce416cd5-54d1-438a-94b7-1cb06d859b6d" containerName="main"
Jul 15 19:59:43.842217 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.842218    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce416cd5-54d1-438a-94b7-1cb06d859b6d" containerName="main"
Jul 15 19:59:43.842379 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.842234    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2036e60-1484-490d-884d-1fd3a29f152a" containerName="main"
Jul 15 19:59:43.842379 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.842243    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2036e60-1484-490d-884d-1fd3a29f152a" containerName="main"
Jul 15 19:59:43.842379 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.842328    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2036e60-1484-490d-884d-1fd3a29f152a" containerName="main"
Jul 15 19:59:43.842379 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.842346    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce416cd5-54d1-438a-94b7-1cb06d859b6d" containerName="main"
Jul 15 19:59:43.845364 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.845345    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:43.849221 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.849203    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-613f78\"/\"default-dockercfg-7w8dd\""
Jul 15 19:59:43.849327 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.849249    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-613f78\"/\"sub-test-model-613f78-kserve-self-signed-certs\""
Jul 15 19:59:43.849327 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.849306    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-613f78\"/\"openshift-service-ca.crt\""
Jul 15 19:59:43.849465 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.849306    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-613f78\"/\"kube-root-ca.crt\""
Jul 15 19:59:43.856144 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.856122    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"]
Jul 15 19:59:43.868381 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.868359    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 19:59:43.940634 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.940601    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-dshm\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:43.940791 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.940653    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-model-cache\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:43.940791 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.940676    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7ef1512-f0c2-4341-87fd-0883d210bf59-tls-certs\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:43.940791 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.940723    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdnlp\" (UniqueName: \"kubernetes.io/projected/c7ef1512-f0c2-4341-87fd-0883d210bf59-kube-api-access-qdnlp\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:43.940791 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.940778    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-home\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:43.940933 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:43.940808    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-tmp-dir\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.041697 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.041660    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-home\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.041876 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.041711    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-tmp-dir\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.041876 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.041746    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-dshm\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.041876 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.041774    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-model-cache\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.041876 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.041794    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7ef1512-f0c2-4341-87fd-0883d210bf59-tls-certs\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.041876 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.041813    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdnlp\" (UniqueName: \"kubernetes.io/projected/c7ef1512-f0c2-4341-87fd-0883d210bf59-kube-api-access-qdnlp\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.042145 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.042114    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-home\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.042199 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.042124    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-model-cache\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.042255 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.042232    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-tmp-dir\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.044101 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.044083    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-dshm\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.044262 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.044245    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7ef1512-f0c2-4341-87fd-0883d210bf59-tls-certs\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.050916 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.050888    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdnlp\" (UniqueName: \"kubernetes.io/projected/c7ef1512-f0c2-4341-87fd-0883d210bf59-kube-api-access-qdnlp\") pod \"sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") " pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.164238 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.164152    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:44.302224 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.302199    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"]
Jul 15 19:59:44.304477 ip-10-0-143-114 kubenswrapper[2560]: W0715 19:59:44.304432    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7ef1512_f0c2_4341_87fd_0883d210bf59.slice/crio-5489b0b28516f48fe89b2421bcce6adcc6061144e47174b8e9b9ac9b94d63681 WatchSource:0}: Error finding container 5489b0b28516f48fe89b2421bcce6adcc6061144e47174b8e9b9ac9b94d63681: Status 404 returned error can't find the container with id 5489b0b28516f48fe89b2421bcce6adcc6061144e47174b8e9b9ac9b94d63681
Jul 15 19:59:44.767367 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.767332    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2" event={"ID":"c7ef1512-f0c2-4341-87fd-0883d210bf59","Type":"ContainerStarted","Data":"d36dfac108790388e37e410f111790f7f3c010156240ba3d8a6746aa0db13cba"}
Jul 15 19:59:44.767579 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.767376    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2" event={"ID":"c7ef1512-f0c2-4341-87fd-0883d210bf59","Type":"ContainerStarted","Data":"5489b0b28516f48fe89b2421bcce6adcc6061144e47174b8e9b9ac9b94d63681"}
Jul 15 19:59:44.787231 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:44.787152    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2" podStartSLOduration=1.787137537 podStartE2EDuration="1.787137537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:44.785473469 +0000 UTC m=+2656.855332868" watchObservedRunningTime="2026-07-15 19:59:44.787137537 +0000 UTC m=+2656.856996935"
Jul 15 19:59:54.165122 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:54.165084    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:54.165632 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:54.165161    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:54.177837 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:54.177813    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 19:59:54.832774 ip-10-0-143-114 kubenswrapper[2560]: I0715 19:59:54.832749    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 20:00:00.135923 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.135889    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75"]
Jul 15 20:00:00.140002 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.139978    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735760-nbkqp"]
Jul 15 20:00:00.140109 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.140090    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75"
Jul 15 20:00:00.142930 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.142912    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bk6kr\""
Jul 15 20:00:00.144141 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.144125    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq"]
Jul 15 20:00:00.144241 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.144228    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-nbkqp"
Jul 15 20:00:00.147587 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.147565    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq"
Jul 15 20:00:00.148057 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.148034    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75"]
Jul 15 20:00:00.151872 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.151844    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq"]
Jul 15 20:00:00.154156 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.154134    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75"
Jul 15 20:00:00.163435 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.163410    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-nbkqp"
Jul 15 20:00:00.163843 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.163574    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735760-nbkqp"]
Jul 15 20:00:00.191419 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.191391    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq"
Jul 15 20:00:00.318105 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.317381    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75"]
Jul 15 20:00:00.318105 ip-10-0-143-114 kubenswrapper[2560]: W0715 20:00:00.317909    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae21ca4c_fbe7_4793_bdd0_679327ff03a4.slice/crio-5559f595634a1802b1f2dd576a48dc105cc436fc5c5171c182b86e94f2f9a496 WatchSource:0}: Error finding container 5559f595634a1802b1f2dd576a48dc105cc436fc5c5171c182b86e94f2f9a496: Status 404 returned error can't find the container with id 5559f595634a1802b1f2dd576a48dc105cc436fc5c5171c182b86e94f2f9a496
Jul 15 20:00:00.369875 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.369852    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq"]
Jul 15 20:00:00.371285 ip-10-0-143-114 kubenswrapper[2560]: W0715 20:00:00.371261    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda7a0c7_e26c_441d_b278_a18861183be5.slice/crio-77ad7e932d0f1d39be56f4eb2696024e5f30e969a57b356df70f652cb62d06a1 WatchSource:0}: Error finding container 77ad7e932d0f1d39be56f4eb2696024e5f30e969a57b356df70f652cb62d06a1: Status 404 returned error can't find the container with id 77ad7e932d0f1d39be56f4eb2696024e5f30e969a57b356df70f652cb62d06a1
Jul 15 20:00:00.549728 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.549705    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735760-nbkqp"]
Jul 15 20:00:00.551720 ip-10-0-143-114 kubenswrapper[2560]: W0715 20:00:00.551693    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5950e2_7ab9_460a_8f0d_5336fa065902.slice/crio-1a5ad3c63d14fc62f7dad7e74cedd3dfa3272f6f7708e782d110d8eccd297c16 WatchSource:0}: Error finding container 1a5ad3c63d14fc62f7dad7e74cedd3dfa3272f6f7708e782d110d8eccd297c16: Status 404 returned error can't find the container with id 1a5ad3c63d14fc62f7dad7e74cedd3dfa3272f6f7708e782d110d8eccd297c16
Jul 15 20:00:00.856279 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.856075    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq" event={"ID":"dda7a0c7-e26c-441d-b278-a18861183be5","Type":"ContainerStarted","Data":"20382e1c531a8003fa868b6ce034ccf110ac6fe6c4a028db38da0de3b99402bc"}
Jul 15 20:00:00.856279 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.856142    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq" event={"ID":"dda7a0c7-e26c-441d-b278-a18861183be5","Type":"ContainerStarted","Data":"77ad7e932d0f1d39be56f4eb2696024e5f30e969a57b356df70f652cb62d06a1"}
Jul 15 20:00:00.857571 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.857543    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-nbkqp" event={"ID":"4f5950e2-7ab9-460a-8f0d-5336fa065902","Type":"ContainerStarted","Data":"1a5ad3c63d14fc62f7dad7e74cedd3dfa3272f6f7708e782d110d8eccd297c16"}
Jul 15 20:00:00.858945 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.858924    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75" event={"ID":"ae21ca4c-fbe7-4793-bdd0-679327ff03a4","Type":"ContainerStarted","Data":"fb4cf75a975e05346294d58d054862b45d5f5777d672453b7c84e24ec26ccae7"}
Jul 15 20:00:00.859020 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.858962    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75" event={"ID":"ae21ca4c-fbe7-4793-bdd0-679327ff03a4","Type":"ContainerStarted","Data":"5559f595634a1802b1f2dd576a48dc105cc436fc5c5171c182b86e94f2f9a496"}
Jul 15 20:00:00.873685 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.873646    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq" podStartSLOduration=0.873633186 podStartE2EDuration="873.633186ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:00.871165467 +0000 UTC m=+2672.941024866" watchObservedRunningTime="2026-07-15 20:00:00.873633186 +0000 UTC m=+2672.943492584"
Jul 15 20:00:00.887417 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:00.887379    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75" podStartSLOduration=0.887366901 podStartE2EDuration="887.366901ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:00.884433886 +0000 UTC m=+2672.954293286" watchObservedRunningTime="2026-07-15 20:00:00.887366901 +0000 UTC m=+2672.957226298"
Jul 15 20:00:01.866258 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:01.866167    2560 generic.go:356] "Generic (PLEG): container finished" podID="dda7a0c7-e26c-441d-b278-a18861183be5" containerID="20382e1c531a8003fa868b6ce034ccf110ac6fe6c4a028db38da0de3b99402bc" exitCode=0
Jul 15 20:00:01.866258 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:01.866250    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq" event={"ID":"dda7a0c7-e26c-441d-b278-a18861183be5","Type":"ContainerDied","Data":"20382e1c531a8003fa868b6ce034ccf110ac6fe6c4a028db38da0de3b99402bc"}
Jul 15 20:00:01.867807 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:01.867779    2560 generic.go:356] "Generic (PLEG): container finished" podID="4f5950e2-7ab9-460a-8f0d-5336fa065902" containerID="3b9a1762d7593c3570d6b593a088c2f65333a0e34a4651d83937ada5b4af187e" exitCode=0
Jul 15 20:00:01.867902 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:01.867861    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-nbkqp" event={"ID":"4f5950e2-7ab9-460a-8f0d-5336fa065902","Type":"ContainerDied","Data":"3b9a1762d7593c3570d6b593a088c2f65333a0e34a4651d83937ada5b4af187e"}
Jul 15 20:00:01.869226 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:01.869203    2560 generic.go:356] "Generic (PLEG): container finished" podID="ae21ca4c-fbe7-4793-bdd0-679327ff03a4" containerID="fb4cf75a975e05346294d58d054862b45d5f5777d672453b7c84e24ec26ccae7" exitCode=0
Jul 15 20:00:01.869337 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:01.869267    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75" event={"ID":"ae21ca4c-fbe7-4793-bdd0-679327ff03a4","Type":"ContainerDied","Data":"fb4cf75a975e05346294d58d054862b45d5f5777d672453b7c84e24ec26ccae7"}
Jul 15 20:00:03.020253 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:03.020227    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75"
Jul 15 20:00:03.037011 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:03.036989    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq"
Jul 15 20:00:03.060606 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:03.060585    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-nbkqp"
Jul 15 20:00:03.881326 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:03.881295    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq"
Jul 15 20:00:03.881559 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:03.881296    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq" event={"ID":"dda7a0c7-e26c-441d-b278-a18861183be5","Type":"ContainerDied","Data":"77ad7e932d0f1d39be56f4eb2696024e5f30e969a57b356df70f652cb62d06a1"}
Jul 15 20:00:03.881559 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:03.881432    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77ad7e932d0f1d39be56f4eb2696024e5f30e969a57b356df70f652cb62d06a1"
Jul 15 20:00:03.882744 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:03.882714    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-nbkqp"
Jul 15 20:00:03.882744 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:03.882719    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735760-nbkqp" event={"ID":"4f5950e2-7ab9-460a-8f0d-5336fa065902","Type":"ContainerDied","Data":"1a5ad3c63d14fc62f7dad7e74cedd3dfa3272f6f7708e782d110d8eccd297c16"}
Jul 15 20:00:03.882940 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:03.882761    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a5ad3c63d14fc62f7dad7e74cedd3dfa3272f6f7708e782d110d8eccd297c16"
Jul 15 20:00:03.884325 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:03.884301    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75" event={"ID":"ae21ca4c-fbe7-4793-bdd0-679327ff03a4","Type":"ContainerDied","Data":"5559f595634a1802b1f2dd576a48dc105cc436fc5c5171c182b86e94f2f9a496"}
Jul 15 20:00:03.884447 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:03.884349    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5559f595634a1802b1f2dd576a48dc105cc436fc5c5171c182b86e94f2f9a496"
Jul 15 20:00:03.884447 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:03.884309    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75"
Jul 15 20:00:18.308750 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:18.308713    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:19.018530 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:19.018489    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"]
Jul 15 20:00:19.162295 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:19.162261    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:27.957495 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:27.957396    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:28.055364 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:28.055309    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:28.592267 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:28.592235    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 20:00:28.598644 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:28.598621    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6vs2_6b44dee7-5ff5-4b39-b708-cc3e1c620758/ovn-acl-logging/0.log"
Jul 15 20:00:34.038788 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.038718    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2" podUID="c7ef1512-f0c2-4341-87fd-0883d210bf59" containerName="main" containerID="cri-o://d36dfac108790388e37e410f111790f7f3c010156240ba3d8a6746aa0db13cba" gracePeriod=45
Jul 15 20:00:34.292036 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.291971    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 20:00:34.392012 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.391974    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-model-cache\") pod \"c7ef1512-f0c2-4341-87fd-0883d210bf59\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") "
Jul 15 20:00:34.392012 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.392017    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-home\") pod \"c7ef1512-f0c2-4341-87fd-0883d210bf59\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") "
Jul 15 20:00:34.392227 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.392076    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-dshm\") pod \"c7ef1512-f0c2-4341-87fd-0883d210bf59\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") "
Jul 15 20:00:34.392227 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.392115    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7ef1512-f0c2-4341-87fd-0883d210bf59-tls-certs\") pod \"c7ef1512-f0c2-4341-87fd-0883d210bf59\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") "
Jul 15 20:00:34.392227 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.392155    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-tmp-dir\") pod \"c7ef1512-f0c2-4341-87fd-0883d210bf59\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") "
Jul 15 20:00:34.392354 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.392225    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdnlp\" (UniqueName: \"kubernetes.io/projected/c7ef1512-f0c2-4341-87fd-0883d210bf59-kube-api-access-qdnlp\") pod \"c7ef1512-f0c2-4341-87fd-0883d210bf59\" (UID: \"c7ef1512-f0c2-4341-87fd-0883d210bf59\") "
Jul 15 20:00:34.392354 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.392311    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-model-cache" (OuterVolumeSpecName: "model-cache") pod "c7ef1512-f0c2-4341-87fd-0883d210bf59" (UID: "c7ef1512-f0c2-4341-87fd-0883d210bf59"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:00:34.392354 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.392329    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-home" (OuterVolumeSpecName: "home") pod "c7ef1512-f0c2-4341-87fd-0883d210bf59" (UID: "c7ef1512-f0c2-4341-87fd-0883d210bf59"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:00:34.392538 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.392408    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c7ef1512-f0c2-4341-87fd-0883d210bf59" (UID: "c7ef1512-f0c2-4341-87fd-0883d210bf59"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:00:34.392602 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.392546    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-model-cache\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:00:34.392602 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.392562    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-home\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:00:34.392602 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.392570    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-tmp-dir\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:00:34.394125 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.394099    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-dshm" (OuterVolumeSpecName: "dshm") pod "c7ef1512-f0c2-4341-87fd-0883d210bf59" (UID: "c7ef1512-f0c2-4341-87fd-0883d210bf59"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:00:34.394298 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.394268    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7ef1512-f0c2-4341-87fd-0883d210bf59-kube-api-access-qdnlp" (OuterVolumeSpecName: "kube-api-access-qdnlp") pod "c7ef1512-f0c2-4341-87fd-0883d210bf59" (UID: "c7ef1512-f0c2-4341-87fd-0883d210bf59"). InnerVolumeSpecName "kube-api-access-qdnlp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:00:34.394298 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.394277    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7ef1512-f0c2-4341-87fd-0883d210bf59-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c7ef1512-f0c2-4341-87fd-0883d210bf59" (UID: "c7ef1512-f0c2-4341-87fd-0883d210bf59"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:00:34.493153 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.493116    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7ef1512-f0c2-4341-87fd-0883d210bf59-dshm\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:00:34.493153 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.493145    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7ef1512-f0c2-4341-87fd-0883d210bf59-tls-certs\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:00:34.493377 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:34.493163    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdnlp\" (UniqueName: \"kubernetes.io/projected/c7ef1512-f0c2-4341-87fd-0883d210bf59-kube-api-access-qdnlp\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:00:35.049915 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:35.049879    2560 generic.go:356] "Generic (PLEG): container finished" podID="c7ef1512-f0c2-4341-87fd-0883d210bf59" containerID="d36dfac108790388e37e410f111790f7f3c010156240ba3d8a6746aa0db13cba" exitCode=0
Jul 15 20:00:35.050411 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:35.049940    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"
Jul 15 20:00:35.050411 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:35.049962    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2" event={"ID":"c7ef1512-f0c2-4341-87fd-0883d210bf59","Type":"ContainerDied","Data":"d36dfac108790388e37e410f111790f7f3c010156240ba3d8a6746aa0db13cba"}
Jul 15 20:00:35.050411 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:35.050006    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2" event={"ID":"c7ef1512-f0c2-4341-87fd-0883d210bf59","Type":"ContainerDied","Data":"5489b0b28516f48fe89b2421bcce6adcc6061144e47174b8e9b9ac9b94d63681"}
Jul 15 20:00:35.050411 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:35.050023    2560 scope.go:117] "RemoveContainer" containerID="d36dfac108790388e37e410f111790f7f3c010156240ba3d8a6746aa0db13cba"
Jul 15 20:00:35.058887 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:35.058870    2560 scope.go:117] "RemoveContainer" containerID="d36dfac108790388e37e410f111790f7f3c010156240ba3d8a6746aa0db13cba"
Jul 15 20:00:35.059130 ip-10-0-143-114 kubenswrapper[2560]: E0715 20:00:35.059110    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d36dfac108790388e37e410f111790f7f3c010156240ba3d8a6746aa0db13cba\": container with ID starting with d36dfac108790388e37e410f111790f7f3c010156240ba3d8a6746aa0db13cba not found: ID does not exist" containerID="d36dfac108790388e37e410f111790f7f3c010156240ba3d8a6746aa0db13cba"
Jul 15 20:00:35.059172 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:35.059138    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d36dfac108790388e37e410f111790f7f3c010156240ba3d8a6746aa0db13cba"} err="failed to get container status \"d36dfac108790388e37e410f111790f7f3c010156240ba3d8a6746aa0db13cba\": rpc error: code = NotFound desc = could not find container \"d36dfac108790388e37e410f111790f7f3c010156240ba3d8a6746aa0db13cba\": container with ID starting with d36dfac108790388e37e410f111790f7f3c010156240ba3d8a6746aa0db13cba not found: ID does not exist"
Jul 15 20:00:35.067604 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:35.067579    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"]
Jul 15 20:00:35.071037 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:35.071015    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/sub-test-model-613f78-kserve-59c8dd6dcc-k2zx2"]
Jul 15 20:00:36.495895 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:36.495865    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7ef1512-f0c2-4341-87fd-0883d210bf59" path="/var/lib/kubelet/pods/c7ef1512-f0c2-4341-87fd-0883d210bf59/volumes"
Jul 15 20:00:38.851854 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.851819    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"]
Jul 15 20:00:38.852206 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.852195    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7ef1512-f0c2-4341-87fd-0883d210bf59" containerName="main"
Jul 15 20:00:38.852256 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.852208    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7ef1512-f0c2-4341-87fd-0883d210bf59" containerName="main"
Jul 15 20:00:38.852256 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.852223    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f5950e2-7ab9-460a-8f0d-5336fa065902" containerName="cleanup"
Jul 15 20:00:38.852256 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.852228    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f5950e2-7ab9-460a-8f0d-5336fa065902" containerName="cleanup"
Jul 15 20:00:38.852256 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.852238    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae21ca4c-fbe7-4793-bdd0-679327ff03a4" containerName="cleanup"
Jul 15 20:00:38.852256 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.852243    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae21ca4c-fbe7-4793-bdd0-679327ff03a4" containerName="cleanup"
Jul 15 20:00:38.852256 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.852249    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda7a0c7-e26c-441d-b278-a18861183be5" containerName="cleanup"
Jul 15 20:00:38.852256 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.852254    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda7a0c7-e26c-441d-b278-a18861183be5" containerName="cleanup"
Jul 15 20:00:38.852483 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.852303    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7ef1512-f0c2-4341-87fd-0883d210bf59" containerName="main"
Jul 15 20:00:38.852483 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.852315    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda7a0c7-e26c-441d-b278-a18861183be5" containerName="cleanup"
Jul 15 20:00:38.852483 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.852321    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae21ca4c-fbe7-4793-bdd0-679327ff03a4" containerName="cleanup"
Jul 15 20:00:38.852483 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.852326    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f5950e2-7ab9-460a-8f0d-5336fa065902" containerName="cleanup"
Jul 15 20:00:38.854520 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.854494    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:38.858623 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.858596    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e8718e\"/\"default-dockercfg-5ggtc\""
Jul 15 20:00:38.859025 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.858719    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e8718e\"/\"kube-root-ca.crt\""
Jul 15 20:00:38.859025 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.858802    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e8718e\"/\"openshift-service-ca.crt\""
Jul 15 20:00:38.859025 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.858800    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e8718e\"/\"rate-test-model-e8718e-kserve-self-signed-certs\""
Jul 15 20:00:38.869430 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.869404    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"]
Jul 15 20:00:38.934787 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.934756    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-dshm\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:38.934787 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.934790    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfxlf\" (UniqueName: \"kubernetes.io/projected/81ddc0a6-26fd-425d-a7b5-4e85b135b023-kube-api-access-mfxlf\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:38.935015 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.934907    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-model-cache\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:38.935015 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.934938    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-home\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:38.935015 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.934959    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-tmp-dir\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:38.935015 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:38.934986    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81ddc0a6-26fd-425d-a7b5-4e85b135b023-tls-certs\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.035924 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.035889    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-dshm\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.035924 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.035927    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfxlf\" (UniqueName: \"kubernetes.io/projected/81ddc0a6-26fd-425d-a7b5-4e85b135b023-kube-api-access-mfxlf\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.036188 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.035975    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-model-cache\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.036188 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.035992    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-home\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.036188 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.036011    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-tmp-dir\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.036188 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.036030    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81ddc0a6-26fd-425d-a7b5-4e85b135b023-tls-certs\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.036397 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.036369    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-home\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.036444 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.036408    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-model-cache\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.036521 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.036473    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-tmp-dir\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.038176 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.038155    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-dshm\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.038405 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.038387    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81ddc0a6-26fd-425d-a7b5-4e85b135b023-tls-certs\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.044478 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.044437    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfxlf\" (UniqueName: \"kubernetes.io/projected/81ddc0a6-26fd-425d-a7b5-4e85b135b023-kube-api-access-mfxlf\") pod \"rate-test-model-e8718e-kserve-75844677c9-k4xlk\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") " pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.171782 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.171680    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:39.313677 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:39.313654    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"]
Jul 15 20:00:39.315908 ip-10-0-143-114 kubenswrapper[2560]: W0715 20:00:39.315879    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81ddc0a6_26fd_425d_a7b5_4e85b135b023.slice/crio-1e1b89fbc48dbf914b91779540f9b020f4c7219cc047c47d152433df3ed083cc WatchSource:0}: Error finding container 1e1b89fbc48dbf914b91779540f9b020f4c7219cc047c47d152433df3ed083cc: Status 404 returned error can't find the container with id 1e1b89fbc48dbf914b91779540f9b020f4c7219cc047c47d152433df3ed083cc
Jul 15 20:00:40.075606 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:40.075564    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:40.077556 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:40.077525    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk" event={"ID":"81ddc0a6-26fd-425d-a7b5-4e85b135b023","Type":"ContainerStarted","Data":"929ae9f1ed71d521f8b11cf1ba9ffdc750d8c8683e02fc26a0c3065d8969be1c"}
Jul 15 20:00:40.077717 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:40.077569    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk" event={"ID":"81ddc0a6-26fd-425d-a7b5-4e85b135b023","Type":"ContainerStarted","Data":"1e1b89fbc48dbf914b91779540f9b020f4c7219cc047c47d152433df3ed083cc"}
Jul 15 20:00:40.095867 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:40.095809    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk" podStartSLOduration=2.095794413 podStartE2EDuration="2.095794413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:40.094792688 +0000 UTC m=+2712.164652085" watchObservedRunningTime="2026-07-15 20:00:40.095794413 +0000 UTC m=+2712.165653810"
Jul 15 20:00:49.172380 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:49.172331    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:49.172822 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:49.172399    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:49.185379 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:49.185350    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:50.141893 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:50.141857    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:00:58.040315 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.040276    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"]
Jul 15 20:00:58.044403 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.044381    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.047217 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.047189    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-613f78\"/\"kube-root-ca.crt\""
Jul 15 20:00:58.047217 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.047204    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-613f78\"/\"openshift-service-ca.crt\""
Jul 15 20:00:58.048373 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.048357    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-613f78\"/\"default-dockercfg-7w8dd\""
Jul 15 20:00:58.048555 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.048533    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-613f78\"/\"rate-test-model-613f78-kserve-self-signed-certs\""
Jul 15 20:00:58.053903 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.053881    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"]
Jul 15 20:00:58.064494 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.064466    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:00:58.214783 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.214743    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa0dd3f-04f4-4230-bdd4-5827749b6582-tls-certs\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.214783 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.214781    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-dshm\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.215051 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.214818    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-home\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.215051 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.214897    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jlgm\" (UniqueName: \"kubernetes.io/projected/0aa0dd3f-04f4-4230-bdd4-5827749b6582-kube-api-access-6jlgm\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.215051 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.214949    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-model-cache\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.215051 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.214972    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-tmp-dir\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.316188 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.316087    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-dshm\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.316188 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.316154    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-home\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.316416 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.316190    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jlgm\" (UniqueName: \"kubernetes.io/projected/0aa0dd3f-04f4-4230-bdd4-5827749b6582-kube-api-access-6jlgm\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.316416 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.316219    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-model-cache\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.316416 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.316238    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-tmp-dir\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.316416 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.316332    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa0dd3f-04f4-4230-bdd4-5827749b6582-tls-certs\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.316674 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.316632    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-home\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.316674 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.316650    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-model-cache\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.316780 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.316712    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-tmp-dir\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.318398 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.318364    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-dshm\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.318721 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.318704    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa0dd3f-04f4-4230-bdd4-5827749b6582-tls-certs\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.325551 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.325526    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jlgm\" (UniqueName: \"kubernetes.io/projected/0aa0dd3f-04f4-4230-bdd4-5827749b6582-kube-api-access-6jlgm\") pod \"rate-test-model-613f78-kserve-7749db547-jbnmw\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") " pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.354537 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.354507    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:00:58.494684 ip-10-0-143-114 kubenswrapper[2560]: W0715 20:00:58.494656    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0aa0dd3f_04f4_4230_bdd4_5827749b6582.slice/crio-9415673a059cc19c4fc5a4635130c2fc8b24ad8e3c712238211d0c2013be88e9 WatchSource:0}: Error finding container 9415673a059cc19c4fc5a4635130c2fc8b24ad8e3c712238211d0c2013be88e9: Status 404 returned error can't find the container with id 9415673a059cc19c4fc5a4635130c2fc8b24ad8e3c712238211d0c2013be88e9
Jul 15 20:00:58.496018 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:58.495998    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"]
Jul 15 20:00:59.176441 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:59.176368    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw" event={"ID":"0aa0dd3f-04f4-4230-bdd4-5827749b6582","Type":"ContainerStarted","Data":"23fa61cb8c13eca3662ceb8e3f70490756f44351bff31c2a3bc8a4b489d31e85"}
Jul 15 20:00:59.176441 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:59.176418    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw" event={"ID":"0aa0dd3f-04f4-4230-bdd4-5827749b6582","Type":"ContainerStarted","Data":"9415673a059cc19c4fc5a4635130c2fc8b24ad8e3c712238211d0c2013be88e9"}
Jul 15 20:00:59.195907 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:00:59.195852    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw" podStartSLOduration=1.195833774 podStartE2EDuration="1.195833774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:59.195098987 +0000 UTC m=+2731.264958384" watchObservedRunningTime="2026-07-15 20:00:59.195833774 +0000 UTC m=+2731.265693170"
Jul 15 20:01:08.355619 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:08.355580    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:01:08.356055 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:08.355642    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:01:08.367797 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:08.367775    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:01:09.239289 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:09.239262    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:01:35.853187 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:35.853156    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:35.906909 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:35.906869    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:40.852246 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:40.852211    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:40.954315 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:40.954277    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:47.555573 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:47.555536    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:48.582676 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:48.582640    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:48.681012 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:48.680977    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:51.384246 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:51.384158    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"]
Jul 15 20:01:51.405135 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:51.405103    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:52.419135 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:52.419104    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"]
Jul 15 20:01:52.444592 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:52.444562    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:01:53.428780 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:53.428747    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7571-140356346203440"]
Jul 15 20:01:53.432287 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:53.432270    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346203440"
Jul 15 20:01:53.434926 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:53.434907    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tsfrm\""
Jul 15 20:01:53.439386 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:53.439361    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7571-140356346203440"]
Jul 15 20:01:53.529812 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:53.529782    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st2g6\" (UniqueName: \"kubernetes.io/projected/1dd4ed3b-7ed3-48a0-ab57-1746be51bc31-kube-api-access-st2g6\") pod \"test-curl-7571-140356346203440\" (UID: \"1dd4ed3b-7ed3-48a0-ab57-1746be51bc31\") " pod="opendatahub/test-curl-7571-140356346203440"
Jul 15 20:01:53.630652 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:53.630615    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st2g6\" (UniqueName: \"kubernetes.io/projected/1dd4ed3b-7ed3-48a0-ab57-1746be51bc31-kube-api-access-st2g6\") pod \"test-curl-7571-140356346203440\" (UID: \"1dd4ed3b-7ed3-48a0-ab57-1746be51bc31\") " pod="opendatahub/test-curl-7571-140356346203440"
Jul 15 20:01:53.639740 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:53.639711    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st2g6\" (UniqueName: \"kubernetes.io/projected/1dd4ed3b-7ed3-48a0-ab57-1746be51bc31-kube-api-access-st2g6\") pod \"test-curl-7571-140356346203440\" (UID: \"1dd4ed3b-7ed3-48a0-ab57-1746be51bc31\") " pod="opendatahub/test-curl-7571-140356346203440"
Jul 15 20:01:53.742193 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:53.742171    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346203440"
Jul 15 20:01:53.866352 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:53.866324    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7571-140356346203440"]
Jul 15 20:01:53.868322 ip-10-0-143-114 kubenswrapper[2560]: W0715 20:01:53.868300    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1dd4ed3b_7ed3_48a0_ab57_1746be51bc31.slice/crio-5c0f2f07efe59933c806afaf7dc32bf37fb34c0c2d13dd441fea048226b14adf WatchSource:0}: Error finding container 5c0f2f07efe59933c806afaf7dc32bf37fb34c0c2d13dd441fea048226b14adf: Status 404 returned error can't find the container with id 5c0f2f07efe59933c806afaf7dc32bf37fb34c0c2d13dd441fea048226b14adf
Jul 15 20:01:53.869580 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:53.869561    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 20:01:54.452070 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:54.452031    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7571-140356346203440" event={"ID":"1dd4ed3b-7ed3-48a0-ab57-1746be51bc31","Type":"ContainerStarted","Data":"5c0f2f07efe59933c806afaf7dc32bf37fb34c0c2d13dd441fea048226b14adf"}
Jul 15 20:01:55.459080 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:55.459045    2560 generic.go:356] "Generic (PLEG): container finished" podID="1dd4ed3b-7ed3-48a0-ab57-1746be51bc31" containerID="f3bae4165053d0dd95b4440144a23d6e0df793f2bdaba53733272755a3e18a9e" exitCode=0
Jul 15 20:01:55.459521 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:55.459126    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7571-140356346203440" event={"ID":"1dd4ed3b-7ed3-48a0-ab57-1746be51bc31","Type":"ContainerDied","Data":"f3bae4165053d0dd95b4440144a23d6e0df793f2bdaba53733272755a3e18a9e"}
Jul 15 20:01:56.596156 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:56.596134    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346203440"
Jul 15 20:01:56.617409 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:56.617380    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7571-140356346203440_1dd4ed3b-7ed3-48a0-ab57-1746be51bc31/test-curl-7571-140356346203440/0.log"
Jul 15 20:01:56.642404 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:56.642378    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7571-140356346203440"]
Jul 15 20:01:56.645681 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:56.645655    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7571-140356346203440"]
Jul 15 20:01:56.660199 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:56.660172    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-st2g6\" (UniqueName: \"kubernetes.io/projected/1dd4ed3b-7ed3-48a0-ab57-1746be51bc31-kube-api-access-st2g6\") pod \"1dd4ed3b-7ed3-48a0-ab57-1746be51bc31\" (UID: \"1dd4ed3b-7ed3-48a0-ab57-1746be51bc31\") "
Jul 15 20:01:56.662310 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:56.662288    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dd4ed3b-7ed3-48a0-ab57-1746be51bc31-kube-api-access-st2g6" (OuterVolumeSpecName: "kube-api-access-st2g6") pod "1dd4ed3b-7ed3-48a0-ab57-1746be51bc31" (UID: "1dd4ed3b-7ed3-48a0-ab57-1746be51bc31"). InnerVolumeSpecName "kube-api-access-st2g6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:01:56.761206 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:56.761167    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st2g6\" (UniqueName: \"kubernetes.io/projected/1dd4ed3b-7ed3-48a0-ab57-1746be51bc31-kube-api-access-st2g6\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:01:57.472730 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:57.472702    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346203440"
Jul 15 20:01:57.472940 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:57.472705    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c0f2f07efe59933c806afaf7dc32bf37fb34c0c2d13dd441fea048226b14adf"
Jul 15 20:01:57.483439 ip-10-0-143-114 kubenswrapper[2560]: E0715 20:01:57.483405    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7571-140356346203440\" is forbidden: User \"system:node:ip-10-0-143-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-114.ec2.internal' and this object" podUID="1dd4ed3b-7ed3-48a0-ab57-1746be51bc31" pod="opendatahub/test-curl-7571-140356346203440"
Jul 15 20:01:58.494598 ip-10-0-143-114 kubenswrapper[2560]: E0715 20:01:58.494558    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7571-140356346203440\" is forbidden: User \"system:node:ip-10-0-143-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-114.ec2.internal' and this object" podUID="1dd4ed3b-7ed3-48a0-ab57-1746be51bc31" pod="opendatahub/test-curl-7571-140356346203440"
Jul 15 20:01:58.495940 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:01:58.495918    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dd4ed3b-7ed3-48a0-ab57-1746be51bc31" path="/var/lib/kubelet/pods/1dd4ed3b-7ed3-48a0-ab57-1746be51bc31/volumes"
Jul 15 20:02:05.751946 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:05.751913    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7571-140356346369920"]
Jul 15 20:02:05.754060 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:05.752340    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dd4ed3b-7ed3-48a0-ab57-1746be51bc31" containerName="test-curl-7571-140356346203440"
Jul 15 20:02:05.754060 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:05.752354    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dd4ed3b-7ed3-48a0-ab57-1746be51bc31" containerName="test-curl-7571-140356346203440"
Jul 15 20:02:05.754060 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:05.752431    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dd4ed3b-7ed3-48a0-ab57-1746be51bc31" containerName="test-curl-7571-140356346203440"
Jul 15 20:02:05.755364 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:05.755347    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:05.758496 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:05.758471    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tsfrm\""
Jul 15 20:02:05.760104 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:05.760054    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7571-140356346369920"]
Jul 15 20:02:05.848680 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:05.848648    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp6ch\" (UniqueName: \"kubernetes.io/projected/0f906be9-8406-4a15-a816-75f5e19d8d81-kube-api-access-bp6ch\") pod \"test-curl-7571-140356346369920\" (UID: \"0f906be9-8406-4a15-a816-75f5e19d8d81\") " pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:05.950075 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:05.950037    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp6ch\" (UniqueName: \"kubernetes.io/projected/0f906be9-8406-4a15-a816-75f5e19d8d81-kube-api-access-bp6ch\") pod \"test-curl-7571-140356346369920\" (UID: \"0f906be9-8406-4a15-a816-75f5e19d8d81\") " pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:05.959200 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:05.959173    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp6ch\" (UniqueName: \"kubernetes.io/projected/0f906be9-8406-4a15-a816-75f5e19d8d81-kube-api-access-bp6ch\") pod \"test-curl-7571-140356346369920\" (UID: \"0f906be9-8406-4a15-a816-75f5e19d8d81\") " pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:06.065908 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.065806    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:06.204744 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.204685    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7571-140356346369920"]
Jul 15 20:02:06.207250 ip-10-0-143-114 kubenswrapper[2560]: W0715 20:02:06.207227    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f906be9_8406_4a15_a816_75f5e19d8d81.slice/crio-17b458140b1ed8f35e40a7e817e7871b72063f2e5145aa80c9fcc25b4a2f80d0 WatchSource:0}: Error finding container 17b458140b1ed8f35e40a7e817e7871b72063f2e5145aa80c9fcc25b4a2f80d0: Status 404 returned error can't find the container with id 17b458140b1ed8f35e40a7e817e7871b72063f2e5145aa80c9fcc25b4a2f80d0
Jul 15 20:02:06.405993 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.405904    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk" podUID="81ddc0a6-26fd-425d-a7b5-4e85b135b023" containerName="main" containerID="cri-o://929ae9f1ed71d521f8b11cf1ba9ffdc750d8c8683e02fc26a0c3065d8969be1c" gracePeriod=45
Jul 15 20:02:06.522965 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.522936    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7571-140356346369920" event={"ID":"0f906be9-8406-4a15-a816-75f5e19d8d81","Type":"ContainerStarted","Data":"17b458140b1ed8f35e40a7e817e7871b72063f2e5145aa80c9fcc25b4a2f80d0"}
Jul 15 20:02:06.524603 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.524573    2560 generic.go:356] "Generic (PLEG): container finished" podID="81ddc0a6-26fd-425d-a7b5-4e85b135b023" containerID="929ae9f1ed71d521f8b11cf1ba9ffdc750d8c8683e02fc26a0c3065d8969be1c" exitCode=0
Jul 15 20:02:06.524705 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.524616    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk" event={"ID":"81ddc0a6-26fd-425d-a7b5-4e85b135b023","Type":"ContainerDied","Data":"929ae9f1ed71d521f8b11cf1ba9ffdc750d8c8683e02fc26a0c3065d8969be1c"}
Jul 15 20:02:06.677333 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.677252    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:02:06.757703 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.757673    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81ddc0a6-26fd-425d-a7b5-4e85b135b023-tls-certs\") pod \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") "
Jul 15 20:02:06.758127 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.757710    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-model-cache\") pod \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") "
Jul 15 20:02:06.758127 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.757759    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-tmp-dir\") pod \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") "
Jul 15 20:02:06.758127 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.757809    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-home\") pod \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") "
Jul 15 20:02:06.758127 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.757848    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-dshm\") pod \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") "
Jul 15 20:02:06.758127 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.757897    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfxlf\" (UniqueName: \"kubernetes.io/projected/81ddc0a6-26fd-425d-a7b5-4e85b135b023-kube-api-access-mfxlf\") pod \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\" (UID: \"81ddc0a6-26fd-425d-a7b5-4e85b135b023\") "
Jul 15 20:02:06.758127 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.758005    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-model-cache" (OuterVolumeSpecName: "model-cache") pod "81ddc0a6-26fd-425d-a7b5-4e85b135b023" (UID: "81ddc0a6-26fd-425d-a7b5-4e85b135b023"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:02:06.758127 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.758086    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "81ddc0a6-26fd-425d-a7b5-4e85b135b023" (UID: "81ddc0a6-26fd-425d-a7b5-4e85b135b023"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:02:06.758527 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.758225    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-model-cache\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:06.758527 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.758238    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-tmp-dir\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:06.758527 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.758390    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-home" (OuterVolumeSpecName: "home") pod "81ddc0a6-26fd-425d-a7b5-4e85b135b023" (UID: "81ddc0a6-26fd-425d-a7b5-4e85b135b023"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:02:06.759934 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.759913    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-dshm" (OuterVolumeSpecName: "dshm") pod "81ddc0a6-26fd-425d-a7b5-4e85b135b023" (UID: "81ddc0a6-26fd-425d-a7b5-4e85b135b023"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:02:06.760004 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.759930    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81ddc0a6-26fd-425d-a7b5-4e85b135b023-kube-api-access-mfxlf" (OuterVolumeSpecName: "kube-api-access-mfxlf") pod "81ddc0a6-26fd-425d-a7b5-4e85b135b023" (UID: "81ddc0a6-26fd-425d-a7b5-4e85b135b023"). InnerVolumeSpecName "kube-api-access-mfxlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:02:06.760148 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.760128    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81ddc0a6-26fd-425d-a7b5-4e85b135b023-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "81ddc0a6-26fd-425d-a7b5-4e85b135b023" (UID: "81ddc0a6-26fd-425d-a7b5-4e85b135b023"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:02:06.859074 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.859034    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-dshm\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:06.859074 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.859066    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfxlf\" (UniqueName: \"kubernetes.io/projected/81ddc0a6-26fd-425d-a7b5-4e85b135b023-kube-api-access-mfxlf\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:06.859074 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.859079    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81ddc0a6-26fd-425d-a7b5-4e85b135b023-tls-certs\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:06.859315 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:06.859091    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81ddc0a6-26fd-425d-a7b5-4e85b135b023-home\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:07.439597 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.439553    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw" podUID="0aa0dd3f-04f4-4230-bdd4-5827749b6582" containerName="main" containerID="cri-o://23fa61cb8c13eca3662ceb8e3f70490756f44351bff31c2a3bc8a4b489d31e85" gracePeriod=45
Jul 15 20:02:07.530917 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.530887    2560 generic.go:356] "Generic (PLEG): container finished" podID="0f906be9-8406-4a15-a816-75f5e19d8d81" containerID="86650a9c342c0fa770d54abc47025104ef7b1a4af23416cb79e25ea157c67274" exitCode=0
Jul 15 20:02:07.531078 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.530984    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7571-140356346369920" event={"ID":"0f906be9-8406-4a15-a816-75f5e19d8d81","Type":"ContainerDied","Data":"86650a9c342c0fa770d54abc47025104ef7b1a4af23416cb79e25ea157c67274"}
Jul 15 20:02:07.532294 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.532270    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk" event={"ID":"81ddc0a6-26fd-425d-a7b5-4e85b135b023","Type":"ContainerDied","Data":"1e1b89fbc48dbf914b91779540f9b020f4c7219cc047c47d152433df3ed083cc"}
Jul 15 20:02:07.532294 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.532283    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"
Jul 15 20:02:07.532438 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.532307    2560 scope.go:117] "RemoveContainer" containerID="929ae9f1ed71d521f8b11cf1ba9ffdc750d8c8683e02fc26a0c3065d8969be1c"
Jul 15 20:02:07.567351 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.567325    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"]
Jul 15 20:02:07.571990 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.571965    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/rate-test-model-e8718e-kserve-75844677c9-k4xlk"]
Jul 15 20:02:07.699545 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.699489    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:02:07.767835 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.767806    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-dshm\") pod \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") "
Jul 15 20:02:07.768193 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.767841    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jlgm\" (UniqueName: \"kubernetes.io/projected/0aa0dd3f-04f4-4230-bdd4-5827749b6582-kube-api-access-6jlgm\") pod \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") "
Jul 15 20:02:07.768193 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.767877    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-model-cache\") pod \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") "
Jul 15 20:02:07.768193 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.767893    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-tmp-dir\") pod \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") "
Jul 15 20:02:07.768193 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.767971    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-home\") pod \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") "
Jul 15 20:02:07.768193 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.768010    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa0dd3f-04f4-4230-bdd4-5827749b6582-tls-certs\") pod \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\" (UID: \"0aa0dd3f-04f4-4230-bdd4-5827749b6582\") "
Jul 15 20:02:07.768193 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.768128    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-model-cache" (OuterVolumeSpecName: "model-cache") pod "0aa0dd3f-04f4-4230-bdd4-5827749b6582" (UID: "0aa0dd3f-04f4-4230-bdd4-5827749b6582"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:02:07.768193 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.768171    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0aa0dd3f-04f4-4230-bdd4-5827749b6582" (UID: "0aa0dd3f-04f4-4230-bdd4-5827749b6582"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:02:07.768539 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.768282    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-home" (OuterVolumeSpecName: "home") pod "0aa0dd3f-04f4-4230-bdd4-5827749b6582" (UID: "0aa0dd3f-04f4-4230-bdd4-5827749b6582"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:02:07.768539 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.768334    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-model-cache\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:07.768539 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.768356    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-tmp-dir\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:07.769997 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.769974    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-dshm" (OuterVolumeSpecName: "dshm") pod "0aa0dd3f-04f4-4230-bdd4-5827749b6582" (UID: "0aa0dd3f-04f4-4230-bdd4-5827749b6582"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 20:02:07.769997 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.769990    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa0dd3f-04f4-4230-bdd4-5827749b6582-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0aa0dd3f-04f4-4230-bdd4-5827749b6582" (UID: "0aa0dd3f-04f4-4230-bdd4-5827749b6582"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 20:02:07.770142 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.770032    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa0dd3f-04f4-4230-bdd4-5827749b6582-kube-api-access-6jlgm" (OuterVolumeSpecName: "kube-api-access-6jlgm") pod "0aa0dd3f-04f4-4230-bdd4-5827749b6582" (UID: "0aa0dd3f-04f4-4230-bdd4-5827749b6582"). InnerVolumeSpecName "kube-api-access-6jlgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:02:07.868962 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.868922    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-home\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:07.868962 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.868955    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa0dd3f-04f4-4230-bdd4-5827749b6582-tls-certs\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:07.868962 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.868967    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa0dd3f-04f4-4230-bdd4-5827749b6582-dshm\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:07.869197 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:07.868976    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jlgm\" (UniqueName: \"kubernetes.io/projected/0aa0dd3f-04f4-4230-bdd4-5827749b6582-kube-api-access-6jlgm\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:08.495945 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.495904    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81ddc0a6-26fd-425d-a7b5-4e85b135b023" path="/var/lib/kubelet/pods/81ddc0a6-26fd-425d-a7b5-4e85b135b023/volumes"
Jul 15 20:02:08.537915 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.537878    2560 generic.go:356] "Generic (PLEG): container finished" podID="0aa0dd3f-04f4-4230-bdd4-5827749b6582" containerID="23fa61cb8c13eca3662ceb8e3f70490756f44351bff31c2a3bc8a4b489d31e85" exitCode=0
Jul 15 20:02:08.537915 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.537913    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"
Jul 15 20:02:08.538159 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.537914    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw" event={"ID":"0aa0dd3f-04f4-4230-bdd4-5827749b6582","Type":"ContainerDied","Data":"23fa61cb8c13eca3662ceb8e3f70490756f44351bff31c2a3bc8a4b489d31e85"}
Jul 15 20:02:08.538159 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.537967    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw" event={"ID":"0aa0dd3f-04f4-4230-bdd4-5827749b6582","Type":"ContainerDied","Data":"9415673a059cc19c4fc5a4635130c2fc8b24ad8e3c712238211d0c2013be88e9"}
Jul 15 20:02:08.538159 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.537990    2560 scope.go:117] "RemoveContainer" containerID="23fa61cb8c13eca3662ceb8e3f70490756f44351bff31c2a3bc8a4b489d31e85"
Jul 15 20:02:08.546786 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.546765    2560 scope.go:117] "RemoveContainer" containerID="23fa61cb8c13eca3662ceb8e3f70490756f44351bff31c2a3bc8a4b489d31e85"
Jul 15 20:02:08.547084 ip-10-0-143-114 kubenswrapper[2560]: E0715 20:02:08.547063    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23fa61cb8c13eca3662ceb8e3f70490756f44351bff31c2a3bc8a4b489d31e85\": container with ID starting with 23fa61cb8c13eca3662ceb8e3f70490756f44351bff31c2a3bc8a4b489d31e85 not found: ID does not exist" containerID="23fa61cb8c13eca3662ceb8e3f70490756f44351bff31c2a3bc8a4b489d31e85"
Jul 15 20:02:08.547142 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.547098    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23fa61cb8c13eca3662ceb8e3f70490756f44351bff31c2a3bc8a4b489d31e85"} err="failed to get container status \"23fa61cb8c13eca3662ceb8e3f70490756f44351bff31c2a3bc8a4b489d31e85\": rpc error: code = NotFound desc = could not find container \"23fa61cb8c13eca3662ceb8e3f70490756f44351bff31c2a3bc8a4b489d31e85\": container with ID starting with 23fa61cb8c13eca3662ceb8e3f70490756f44351bff31c2a3bc8a4b489d31e85 not found: ID does not exist"
Jul 15 20:02:08.556272 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.555880    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"]
Jul 15 20:02:08.557956 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.557933    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-613f78/rate-test-model-613f78-kserve-7749db547-jbnmw"]
Jul 15 20:02:08.676772 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.676745    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:08.698868 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.698838    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7571-140356346369920_0f906be9-8406-4a15-a816-75f5e19d8d81/test-curl-7571-140356346369920/0.log"
Jul 15 20:02:08.724115 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.724081    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7571-140356346369920"]
Jul 15 20:02:08.728334 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.728307    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7571-140356346369920"]
Jul 15 20:02:08.778138 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.778046    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp6ch\" (UniqueName: \"kubernetes.io/projected/0f906be9-8406-4a15-a816-75f5e19d8d81-kube-api-access-bp6ch\") pod \"0f906be9-8406-4a15-a816-75f5e19d8d81\" (UID: \"0f906be9-8406-4a15-a816-75f5e19d8d81\") "
Jul 15 20:02:08.780256 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.780219    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f906be9-8406-4a15-a816-75f5e19d8d81-kube-api-access-bp6ch" (OuterVolumeSpecName: "kube-api-access-bp6ch") pod "0f906be9-8406-4a15-a816-75f5e19d8d81" (UID: "0f906be9-8406-4a15-a816-75f5e19d8d81"). InnerVolumeSpecName "kube-api-access-bp6ch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:02:08.853629 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.853589    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7571-140356346369920"]
Jul 15 20:02:08.853986 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.853969    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f906be9-8406-4a15-a816-75f5e19d8d81" containerName="test-curl-7571-140356346369920"
Jul 15 20:02:08.853986 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.853984    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f906be9-8406-4a15-a816-75f5e19d8d81" containerName="test-curl-7571-140356346369920"
Jul 15 20:02:08.854142 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.853995    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81ddc0a6-26fd-425d-a7b5-4e85b135b023" containerName="main"
Jul 15 20:02:08.854142 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.854001    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="81ddc0a6-26fd-425d-a7b5-4e85b135b023" containerName="main"
Jul 15 20:02:08.854142 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.854026    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa0dd3f-04f4-4230-bdd4-5827749b6582" containerName="main"
Jul 15 20:02:08.854142 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.854031    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa0dd3f-04f4-4230-bdd4-5827749b6582" containerName="main"
Jul 15 20:02:08.854142 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.854095    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="81ddc0a6-26fd-425d-a7b5-4e85b135b023" containerName="main"
Jul 15 20:02:08.854142 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.854103    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa0dd3f-04f4-4230-bdd4-5827749b6582" containerName="main"
Jul 15 20:02:08.854142 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.854111    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f906be9-8406-4a15-a816-75f5e19d8d81" containerName="test-curl-7571-140356346369920"
Jul 15 20:02:08.857062 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.857044    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:08.871047 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.866640    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7571-140356346369920"]
Jul 15 20:02:08.878754 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.878728    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp6ch\" (UniqueName: \"kubernetes.io/projected/0f906be9-8406-4a15-a816-75f5e19d8d81-kube-api-access-bp6ch\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:08.980029 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:08.979987    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9msp\" (UniqueName: \"kubernetes.io/projected/f59aa6f3-6dd4-4fd3-b754-702ee8ce0551-kube-api-access-m9msp\") pod \"test-curl-7571-140356346369920\" (UID: \"f59aa6f3-6dd4-4fd3-b754-702ee8ce0551\") " pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:09.081189 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:09.081094    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9msp\" (UniqueName: \"kubernetes.io/projected/f59aa6f3-6dd4-4fd3-b754-702ee8ce0551-kube-api-access-m9msp\") pod \"test-curl-7571-140356346369920\" (UID: \"f59aa6f3-6dd4-4fd3-b754-702ee8ce0551\") " pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:09.089970 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:09.089937    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9msp\" (UniqueName: \"kubernetes.io/projected/f59aa6f3-6dd4-4fd3-b754-702ee8ce0551-kube-api-access-m9msp\") pod \"test-curl-7571-140356346369920\" (UID: \"f59aa6f3-6dd4-4fd3-b754-702ee8ce0551\") " pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:09.167810 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:09.167770    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:09.304725 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:09.304694    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7571-140356346369920"]
Jul 15 20:02:09.306374 ip-10-0-143-114 kubenswrapper[2560]: W0715 20:02:09.306340    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf59aa6f3_6dd4_4fd3_b754_702ee8ce0551.slice/crio-7fd94a0a728fb8110c428e974a4e4db2801e750dd83dd839059fab6efc9f80b9 WatchSource:0}: Error finding container 7fd94a0a728fb8110c428e974a4e4db2801e750dd83dd839059fab6efc9f80b9: Status 404 returned error can't find the container with id 7fd94a0a728fb8110c428e974a4e4db2801e750dd83dd839059fab6efc9f80b9
Jul 15 20:02:09.545760 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:09.545730    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:09.545930 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:09.545728    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17b458140b1ed8f35e40a7e817e7871b72063f2e5145aa80c9fcc25b4a2f80d0"
Jul 15 20:02:09.546864 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:09.546827    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7571-140356346369920" event={"ID":"f59aa6f3-6dd4-4fd3-b754-702ee8ce0551","Type":"ContainerStarted","Data":"7fd94a0a728fb8110c428e974a4e4db2801e750dd83dd839059fab6efc9f80b9"}
Jul 15 20:02:09.559065 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:09.559036    2560 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7571-140356346369920" oldPodUID="0f906be9-8406-4a15-a816-75f5e19d8d81" podUID="f59aa6f3-6dd4-4fd3-b754-702ee8ce0551"
Jul 15 20:02:10.495984 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:10.495954    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aa0dd3f-04f4-4230-bdd4-5827749b6582" path="/var/lib/kubelet/pods/0aa0dd3f-04f4-4230-bdd4-5827749b6582/volumes"
Jul 15 20:02:10.496344 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:10.496255    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f906be9-8406-4a15-a816-75f5e19d8d81" path="/var/lib/kubelet/pods/0f906be9-8406-4a15-a816-75f5e19d8d81/volumes"
Jul 15 20:02:10.552816 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:10.552779    2560 generic.go:356] "Generic (PLEG): container finished" podID="f59aa6f3-6dd4-4fd3-b754-702ee8ce0551" containerID="2f23215bd49e346f01930b431cc90d31dc47b75aee86b82a5141dd19d64e8f9a" exitCode=0
Jul 15 20:02:10.552984 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:10.552871    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7571-140356346369920" event={"ID":"f59aa6f3-6dd4-4fd3-b754-702ee8ce0551","Type":"ContainerDied","Data":"2f23215bd49e346f01930b431cc90d31dc47b75aee86b82a5141dd19d64e8f9a"}
Jul 15 20:02:11.694270 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.694247    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:11.713892 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.713819    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7571-140356346369920_f59aa6f3-6dd4-4fd3-b754-702ee8ce0551/test-curl-7571-140356346369920/0.log"
Jul 15 20:02:11.734640 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.734608    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7571-140356346369920"]
Jul 15 20:02:11.740406 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.740384    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7571-140356346369920"]
Jul 15 20:02:11.807592 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.807560    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9msp\" (UniqueName: \"kubernetes.io/projected/f59aa6f3-6dd4-4fd3-b754-702ee8ce0551-kube-api-access-m9msp\") pod \"f59aa6f3-6dd4-4fd3-b754-702ee8ce0551\" (UID: \"f59aa6f3-6dd4-4fd3-b754-702ee8ce0551\") "
Jul 15 20:02:11.809660 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.809597    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f59aa6f3-6dd4-4fd3-b754-702ee8ce0551-kube-api-access-m9msp" (OuterVolumeSpecName: "kube-api-access-m9msp") pod "f59aa6f3-6dd4-4fd3-b754-702ee8ce0551" (UID: "f59aa6f3-6dd4-4fd3-b754-702ee8ce0551"). InnerVolumeSpecName "kube-api-access-m9msp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 20:02:11.908402 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.908362    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9msp\" (UniqueName: \"kubernetes.io/projected/f59aa6f3-6dd4-4fd3-b754-702ee8ce0551-kube-api-access-m9msp\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:11.932934 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.932898    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8"]
Jul 15 20:02:11.933281 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.933269    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f59aa6f3-6dd4-4fd3-b754-702ee8ce0551" containerName="test-curl-7571-140356346369920"
Jul 15 20:02:11.933324 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.933283    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f59aa6f3-6dd4-4fd3-b754-702ee8ce0551" containerName="test-curl-7571-140356346369920"
Jul 15 20:02:11.933356 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.933350    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f59aa6f3-6dd4-4fd3-b754-702ee8ce0551" containerName="test-curl-7571-140356346369920"
Jul 15 20:02:11.936364 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.936349    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8"
Jul 15 20:02:11.939064 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.939042    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bk6kr\""
Jul 15 20:02:11.939209 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.939073    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 20:02:11.944356 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:11.944330    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8"]
Jul 15 20:02:12.008902 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:12.008867    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3833123-76ea-41fb-880d-1bae767101b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8\" (UID: \"d3833123-76ea-41fb-880d-1bae767101b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8"
Jul 15 20:02:12.109761 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:12.109658    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3833123-76ea-41fb-880d-1bae767101b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8\" (UID: \"d3833123-76ea-41fb-880d-1bae767101b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8"
Jul 15 20:02:12.110313 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:12.110286    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3833123-76ea-41fb-880d-1bae767101b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8\" (UID: \"d3833123-76ea-41fb-880d-1bae767101b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8"
Jul 15 20:02:12.249139 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:12.249085    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8"
Jul 15 20:02:12.375795 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:12.375771    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8"]
Jul 15 20:02:12.376741 ip-10-0-143-114 kubenswrapper[2560]: W0715 20:02:12.376719    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3833123_76ea_41fb_880d_1bae767101b7.slice/crio-8e843c219ac1eab5d27d6860300c2082027ce743ebaea5d859d899161698c409 WatchSource:0}: Error finding container 8e843c219ac1eab5d27d6860300c2082027ce743ebaea5d859d899161698c409: Status 404 returned error can't find the container with id 8e843c219ac1eab5d27d6860300c2082027ce743ebaea5d859d899161698c409
Jul 15 20:02:12.495639 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:12.495608    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f59aa6f3-6dd4-4fd3-b754-702ee8ce0551" path="/var/lib/kubelet/pods/f59aa6f3-6dd4-4fd3-b754-702ee8ce0551/volumes"
Jul 15 20:02:12.564915 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:12.564879    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8" event={"ID":"d3833123-76ea-41fb-880d-1bae767101b7","Type":"ContainerStarted","Data":"8e843c219ac1eab5d27d6860300c2082027ce743ebaea5d859d899161698c409"}
Jul 15 20:02:12.566216 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:12.566194    2560 scope.go:117] "RemoveContainer" containerID="2f23215bd49e346f01930b431cc90d31dc47b75aee86b82a5141dd19d64e8f9a"
Jul 15 20:02:12.566363 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:12.566228    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346369920"
Jul 15 20:02:12.582789 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:12.582764    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:12.682230 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:12.682196    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:13.574273 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:13.574240    2560 generic.go:356] "Generic (PLEG): container finished" podID="d3833123-76ea-41fb-880d-1bae767101b7" containerID="ec252badcff8f0663ecc9ac195f003f842ea96bdccf50e73158bcaf34aa29fdf" exitCode=0
Jul 15 20:02:13.574692 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:13.574329    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8" event={"ID":"d3833123-76ea-41fb-880d-1bae767101b7","Type":"ContainerDied","Data":"ec252badcff8f0663ecc9ac195f003f842ea96bdccf50e73158bcaf34aa29fdf"}
Jul 15 20:02:14.709258 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:14.709235    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8"
Jul 15 20:02:14.835543 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:14.835439    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3833123-76ea-41fb-880d-1bae767101b7-maas-api-service-ca\") pod \"d3833123-76ea-41fb-880d-1bae767101b7\" (UID: \"d3833123-76ea-41fb-880d-1bae767101b7\") "
Jul 15 20:02:14.835786 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:14.835761    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3833123-76ea-41fb-880d-1bae767101b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d3833123-76ea-41fb-880d-1bae767101b7" (UID: "d3833123-76ea-41fb-880d-1bae767101b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:02:14.936237 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:14.936198    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3833123-76ea-41fb-880d-1bae767101b7-maas-api-service-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:15.584660 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:15.584629    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8" event={"ID":"d3833123-76ea-41fb-880d-1bae767101b7","Type":"ContainerDied","Data":"8e843c219ac1eab5d27d6860300c2082027ce743ebaea5d859d899161698c409"}
Jul 15 20:02:15.584845 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:15.584665    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8"
Jul 15 20:02:15.584845 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:15.584685    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e843c219ac1eab5d27d6860300c2082027ce743ebaea5d859d899161698c409"
Jul 15 20:02:22.389756 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:22.389722    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75"]
Jul 15 20:02:22.392007 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:22.391977    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:22.392126 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:22.392034    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-e8718e-29735760-c7m75"]
Jul 15 20:02:22.496350 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:22.496315    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae21ca4c-fbe7-4793-bdd0-679327ff03a4" path="/var/lib/kubelet/pods/ae21ca4c-fbe7-4793-bdd0-679327ff03a4/volumes"
Jul 15 20:02:27.030569 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:27.030532    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:27.437150 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:27.437059    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:33.231557 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:33.231522    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:33.332439 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:33.332400    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:33.482842 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:33.482731    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:33.715896 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:33.715856    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:33.852314 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:33.852282    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59"]
Jul 15 20:02:33.852679 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:33.852665    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3833123-76ea-41fb-880d-1bae767101b7" containerName="revoke-keys"
Jul 15 20:02:33.852733 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:33.852680    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3833123-76ea-41fb-880d-1bae767101b7" containerName="revoke-keys"
Jul 15 20:02:33.852772 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:33.852751    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3833123-76ea-41fb-880d-1bae767101b7" containerName="revoke-keys"
Jul 15 20:02:33.858399 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:33.858382    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59"
Jul 15 20:02:33.861223 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:33.861031    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 20:02:33.861223 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:33.861131    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bk6kr\""
Jul 15 20:02:33.862239 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:33.862215    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59"]
Jul 15 20:02:33.908817 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:33.908785    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e0b3b3f-e218-4c6f-a6c6-1f514735db93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-613f78-k4q59\" (UID: \"4e0b3b3f-e218-4c6f-a6c6-1f514735db93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59"
Jul 15 20:02:34.009596 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:34.009561    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e0b3b3f-e218-4c6f-a6c6-1f514735db93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-613f78-k4q59\" (UID: \"4e0b3b3f-e218-4c6f-a6c6-1f514735db93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59"
Jul 15 20:02:34.010254 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:34.010232    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e0b3b3f-e218-4c6f-a6c6-1f514735db93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-613f78-k4q59\" (UID: \"4e0b3b3f-e218-4c6f-a6c6-1f514735db93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59"
Jul 15 20:02:34.170669 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:34.170583    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59"
Jul 15 20:02:34.306503 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:34.306479    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59"]
Jul 15 20:02:34.308705 ip-10-0-143-114 kubenswrapper[2560]: W0715 20:02:34.308674    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e0b3b3f_e218_4c6f_a6c6_1f514735db93.slice/crio-09d74b36391588bcad2fe50ee3905222f3d165dda0d368bd5ce9ff0a996292fa WatchSource:0}: Error finding container 09d74b36391588bcad2fe50ee3905222f3d165dda0d368bd5ce9ff0a996292fa: Status 404 returned error can't find the container with id 09d74b36391588bcad2fe50ee3905222f3d165dda0d368bd5ce9ff0a996292fa
Jul 15 20:02:34.351392 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:34.351358    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:34.452031 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:34.451942    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:34.681642 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:34.681609    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59" event={"ID":"4e0b3b3f-e218-4c6f-a6c6-1f514735db93","Type":"ContainerStarted","Data":"ea9b67b73a00f97b74d079e454c9edb56eb281e822963c64677c9b2f9057996f"}
Jul 15 20:02:34.681830 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:34.681661    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59" event={"ID":"4e0b3b3f-e218-4c6f-a6c6-1f514735db93","Type":"ContainerStarted","Data":"09d74b36391588bcad2fe50ee3905222f3d165dda0d368bd5ce9ff0a996292fa"}
Jul 15 20:02:34.698079 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:34.698028    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59" podStartSLOduration=1.6980118480000002 podStartE2EDuration="1.698011848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:02:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:02:34.696475367 +0000 UTC m=+2826.766334766" watchObservedRunningTime="2026-07-15 20:02:34.698011848 +0000 UTC m=+2826.767871237"
Jul 15 20:02:35.688630 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:35.688594    2560 generic.go:356] "Generic (PLEG): container finished" podID="4e0b3b3f-e218-4c6f-a6c6-1f514735db93" containerID="ea9b67b73a00f97b74d079e454c9edb56eb281e822963c64677c9b2f9057996f" exitCode=0
Jul 15 20:02:35.689005 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:35.688676    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59" event={"ID":"4e0b3b3f-e218-4c6f-a6c6-1f514735db93","Type":"ContainerDied","Data":"ea9b67b73a00f97b74d079e454c9edb56eb281e822963c64677c9b2f9057996f"}
Jul 15 20:02:36.823989 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:36.823960    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59"
Jul 15 20:02:36.836319 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:36.836298    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e0b3b3f-e218-4c6f-a6c6-1f514735db93-maas-api-service-ca\") pod \"4e0b3b3f-e218-4c6f-a6c6-1f514735db93\" (UID: \"4e0b3b3f-e218-4c6f-a6c6-1f514735db93\") "
Jul 15 20:02:36.836626 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:36.836604    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e0b3b3f-e218-4c6f-a6c6-1f514735db93-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e0b3b3f-e218-4c6f-a6c6-1f514735db93" (UID: "4e0b3b3f-e218-4c6f-a6c6-1f514735db93"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 20:02:36.937538 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:36.937506    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e0b3b3f-e218-4c6f-a6c6-1f514735db93-maas-api-service-ca\") on node \"ip-10-0-143-114.ec2.internal\" DevicePath \"\""
Jul 15 20:02:37.701687 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:37.701657    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59"
Jul 15 20:02:37.701858 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:37.701692    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-613f78-k4q59" event={"ID":"4e0b3b3f-e218-4c6f-a6c6-1f514735db93","Type":"ContainerDied","Data":"09d74b36391588bcad2fe50ee3905222f3d165dda0d368bd5ce9ff0a996292fa"}
Jul 15 20:02:37.701858 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:37.701743    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09d74b36391588bcad2fe50ee3905222f3d165dda0d368bd5ce9ff0a996292fa"
Jul 15 20:02:44.383589 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:44.383548    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:44.384018 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:44.383923    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq"]
Jul 15 20:02:44.385847 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:44.385824    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-613f78-29735760-4wpjq"]
Jul 15 20:02:44.494772 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:44.494737    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dda7a0c7-e26c-441d-b278-a18861183be5" path="/var/lib/kubelet/pods/dda7a0c7-e26c-441d-b278-a18861183be5/volumes"
Jul 15 20:02:48.942014 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:48.941975    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:49.323649 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:49.323616    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:55.472253 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:55.472212    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:55.542037 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:55.542005    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:55.756088 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:55.756047    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:02:56.007213 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:02:56.007111    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:02.258127 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:02.258089    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-s8vxc_acf759f2-50d0-45f3-a138-d8919ea53d58/manager/0.log"
Jul 15 20:03:02.543729 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:02.543646    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-78lfd_230e7abb-5a9a-422e-9302-d9d2091ee70a/server/0.log"
Jul 15 20:03:02.673528 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:02.673498    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mm4c8_5a570eb6-6ef3-49c2-9f97-620668fcef9e/manager/0.log"
Jul 15 20:03:02.795678 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:02.795595    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f46f5478c-m22fw_e411a261-a349-4b1c-9f8a-0269bef246b6/copy-manifests/0.log"
Jul 15 20:03:02.803514 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:02.803484    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f46f5478c-m22fw_e411a261-a349-4b1c-9f8a-0269bef246b6/manager/0.log"
Jul 15 20:03:03.797504 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:03.797476    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735760-nbkqp_4f5950e2-7ab9-460a-8f0d-5336fa065902/cleanup/0.log"
Jul 15 20:03:04.139222 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:04.139137    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-06b9cb-wxrw7_d17990a8-c2a4-453a-9b95-14dfebe13c89/revoke-keys/0.log"
Jul 15 20:03:04.250901 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:04.250873    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-a4ccbf-mdnll_24d18cd0-cfb9-4374-95e0-7159f82e3e2a/revoke-keys/0.log"
Jul 15 20:03:04.480121 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:04.480034    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-da0342-c9s8d_baecea79-9943-4766-9212-aa46ff27f4d9/revoke-keys/0.log"
Jul 15 20:03:04.605079 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:04.605052    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-435e1e-tkr2m_5bf4b6bf-5c88-464a-af9d-7a64649889ec/revoke-keys/0.log"
Jul 15 20:03:04.954173 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:04.954142    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-f3e285c6-ghlgg_beadb106-5009-4a80-b208-3be02f16f80c/revoke-keys/0.log"
Jul 15 20:03:05.071182 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:05.071152    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-e8718e-mr9x8_d3833123-76ea-41fb-880d-1bae767101b7/revoke-keys/0.log"
Jul 15 20:03:05.185964 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:05.185933    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-613f78-k4q59_4e0b3b3f-e218-4c6f-a6c6-1f514735db93/revoke-keys/0.log"
Jul 15 20:03:07.287915 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:07.287883    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bkkwx_29b660a7-e824-4e1b-a142-c8b6b47df2a9/manager/0.log"
Jul 15 20:03:08.144807 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:08.144776    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59b767f7ff-8s2cv_136d57a8-1783-4d0d-9a15-77b54b17b3e2/kube-auth-proxy/0.log"
Jul 15 20:03:09.133710 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:09.133678    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-f7jkw_c35ec128-1999-4a20-95f2-f0961bd55d11/main/0.log"
Jul 15 20:03:09.365511 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:09.365478    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-4nx8p_89a2ec48-4d06-45f9-8cbd-a30144708001/main/0.log"
Jul 15 20:03:09.606181 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:09.606149    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-trkj6_1f1c87d3-9a8e-46df-aa1e-b0908b77650c/main/0.log"
Jul 15 20:03:15.238015 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.237979    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pp6mc/must-gather-6t8zb"]
Jul 15 20:03:15.238393 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.238347    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e0b3b3f-e218-4c6f-a6c6-1f514735db93" containerName="revoke-keys"
Jul 15 20:03:15.238393 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.238357    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e0b3b3f-e218-4c6f-a6c6-1f514735db93" containerName="revoke-keys"
Jul 15 20:03:15.238496 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.238446    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e0b3b3f-e218-4c6f-a6c6-1f514735db93" containerName="revoke-keys"
Jul 15 20:03:15.241620 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.241603    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pp6mc/must-gather-6t8zb"
Jul 15 20:03:15.244575 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.244547    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pp6mc\"/\"kube-root-ca.crt\""
Jul 15 20:03:15.245620 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.245605    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pp6mc\"/\"default-dockercfg-z9m9k\""
Jul 15 20:03:15.245721 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.245606    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pp6mc\"/\"openshift-service-ca.crt\""
Jul 15 20:03:15.258146 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.258126    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pp6mc/must-gather-6t8zb"]
Jul 15 20:03:15.268861 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.268827    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:15.296238 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.296204    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm4z7\" (UniqueName: \"kubernetes.io/projected/e6d08a7f-a5cb-495b-896d-db31f8946d5c-kube-api-access-tm4z7\") pod \"must-gather-6t8zb\" (UID: \"e6d08a7f-a5cb-495b-896d-db31f8946d5c\") " pod="openshift-must-gather-pp6mc/must-gather-6t8zb"
Jul 15 20:03:15.296238 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.296239    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e6d08a7f-a5cb-495b-896d-db31f8946d5c-must-gather-output\") pod \"must-gather-6t8zb\" (UID: \"e6d08a7f-a5cb-495b-896d-db31f8946d5c\") " pod="openshift-must-gather-pp6mc/must-gather-6t8zb"
Jul 15 20:03:15.360994 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.360962    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:15.397435 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.397395    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm4z7\" (UniqueName: \"kubernetes.io/projected/e6d08a7f-a5cb-495b-896d-db31f8946d5c-kube-api-access-tm4z7\") pod \"must-gather-6t8zb\" (UID: \"e6d08a7f-a5cb-495b-896d-db31f8946d5c\") " pod="openshift-must-gather-pp6mc/must-gather-6t8zb"
Jul 15 20:03:15.397435 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.397436    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e6d08a7f-a5cb-495b-896d-db31f8946d5c-must-gather-output\") pod \"must-gather-6t8zb\" (UID: \"e6d08a7f-a5cb-495b-896d-db31f8946d5c\") " pod="openshift-must-gather-pp6mc/must-gather-6t8zb"
Jul 15 20:03:15.397842 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.397823    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e6d08a7f-a5cb-495b-896d-db31f8946d5c-must-gather-output\") pod \"must-gather-6t8zb\" (UID: \"e6d08a7f-a5cb-495b-896d-db31f8946d5c\") " pod="openshift-must-gather-pp6mc/must-gather-6t8zb"
Jul 15 20:03:15.406998 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.406963    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm4z7\" (UniqueName: \"kubernetes.io/projected/e6d08a7f-a5cb-495b-896d-db31f8946d5c-kube-api-access-tm4z7\") pod \"must-gather-6t8zb\" (UID: \"e6d08a7f-a5cb-495b-896d-db31f8946d5c\") " pod="openshift-must-gather-pp6mc/must-gather-6t8zb"
Jul 15 20:03:15.561666 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.561573    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pp6mc/must-gather-6t8zb"
Jul 15 20:03:15.697484 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.697435    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pp6mc/must-gather-6t8zb"]
Jul 15 20:03:15.701065 ip-10-0-143-114 kubenswrapper[2560]: W0715 20:03:15.701031    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6d08a7f_a5cb_495b_896d_db31f8946d5c.slice/crio-d808d2569fc9256e4b2c8b421cf8864881a9911ad4e5fbf34d2de659717a46f2 WatchSource:0}: Error finding container d808d2569fc9256e4b2c8b421cf8864881a9911ad4e5fbf34d2de659717a46f2: Status 404 returned error can't find the container with id d808d2569fc9256e4b2c8b421cf8864881a9911ad4e5fbf34d2de659717a46f2
Jul 15 20:03:15.900944 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:15.900856    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pp6mc/must-gather-6t8zb" event={"ID":"e6d08a7f-a5cb-495b-896d-db31f8946d5c","Type":"ContainerStarted","Data":"d808d2569fc9256e4b2c8b421cf8864881a9911ad4e5fbf34d2de659717a46f2"}
Jul 15 20:03:16.908795 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:16.908694    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pp6mc/must-gather-6t8zb" event={"ID":"e6d08a7f-a5cb-495b-896d-db31f8946d5c","Type":"ContainerStarted","Data":"a5e30dbc5f285feb3b586ac0f40bc9c8b04aa5a4290c7692ce51304720148b8a"}
Jul 15 20:03:16.908795 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:16.908748    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pp6mc/must-gather-6t8zb" event={"ID":"e6d08a7f-a5cb-495b-896d-db31f8946d5c","Type":"ContainerStarted","Data":"d58e33fbd576a8e9b727afd533560be00b688c50a25ffbb9127b9b0556a2b576"}
Jul 15 20:03:16.927111 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:16.927048    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pp6mc/must-gather-6t8zb" podStartSLOduration=0.994199483 podStartE2EDuration="1.927004867s" totalImagesPullingTime="932.805384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:03:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:03:16.9269467 +0000 UTC m=+2868.996806100" watchObservedRunningTime="2026-07-15 20:03:16.927004867 +0000 UTC m=+2868.996864266"
Jul 15 20:03:18.270076 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:18.270048    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-djwqm_995ebd78-54f8-4755-8261-bdca88870130/global-pull-secret-syncer/0.log"
Jul 15 20:03:18.389075 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:18.389045    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log"
Jul 15 20:03:18.569763 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:18.569685    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-114.ec2.internal_ef87f315cd7dcd1a6cbfa6284e94d05d/haproxy/0.log"
Jul 15 20:03:23.968480 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:23.967477    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bkkwx_29b660a7-e824-4e1b-a142-c8b6b47df2a9/manager/0.log"
Jul 15 20:03:25.918182 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:25.918145    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-m5zgx_1c88ecb0-35f5-46fb-8b64-31689b50e178/kube-state-metrics/0.log"
Jul 15 20:03:25.945476 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:25.945436    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-m5zgx_1c88ecb0-35f5-46fb-8b64-31689b50e178/kube-rbac-proxy-main/0.log"
Jul 15 20:03:25.969864 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:25.969836    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-m5zgx_1c88ecb0-35f5-46fb-8b64-31689b50e178/kube-rbac-proxy-self/0.log"
Jul 15 20:03:26.004283 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.004254    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-66c7ff6b65-r8lz2_e61fadd8-c0be-4c32-8b4e-1c660e0549ba/metrics-server/0.log"
Jul 15 20:03:26.035759 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.035681    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-vmnjq_3935f491-571f-44b7-88d0-8d987b3508bc/monitoring-plugin/0.log"
Jul 15 20:03:26.160609 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.160505    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rfhxr_38246ac2-0ceb-4aef-a07b-50dbc742e79d/node-exporter/0.log"
Jul 15 20:03:26.192163 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.192138    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rfhxr_38246ac2-0ceb-4aef-a07b-50dbc742e79d/kube-rbac-proxy/0.log"
Jul 15 20:03:26.216508 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.216481    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rfhxr_38246ac2-0ceb-4aef-a07b-50dbc742e79d/init-textfile/0.log"
Jul 15 20:03:26.338977 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.338934    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-hdm2p_3b0da1ee-8857-46b5-a35d-630cb0817dfe/kube-rbac-proxy-main/0.log"
Jul 15 20:03:26.361518 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.361492    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-hdm2p_3b0da1ee-8857-46b5-a35d-630cb0817dfe/kube-rbac-proxy-self/0.log"
Jul 15 20:03:26.386700 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.386596    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-hdm2p_3b0da1ee-8857-46b5-a35d-630cb0817dfe/openshift-state-metrics/0.log"
Jul 15 20:03:26.426048 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.425963    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e473734-b386-4631-bb86-ef3e295238f9/prometheus/0.log"
Jul 15 20:03:26.451781 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.451751    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e473734-b386-4631-bb86-ef3e295238f9/config-reloader/0.log"
Jul 15 20:03:26.480557 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.480532    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e473734-b386-4631-bb86-ef3e295238f9/thanos-sidecar/0.log"
Jul 15 20:03:26.506085 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.506057    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e473734-b386-4631-bb86-ef3e295238f9/kube-rbac-proxy-web/0.log"
Jul 15 20:03:26.529324 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.529299    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e473734-b386-4631-bb86-ef3e295238f9/kube-rbac-proxy/0.log"
Jul 15 20:03:26.552667 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.552641    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e473734-b386-4631-bb86-ef3e295238f9/kube-rbac-proxy-thanos/0.log"
Jul 15 20:03:26.577566 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.577540    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e473734-b386-4631-bb86-ef3e295238f9/init-config-reloader/0.log"
Jul 15 20:03:26.611323 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.611294    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-nfqd2_a851c47d-8ee9-414c-bded-b74bb2105be5/prometheus-operator/0.log"
Jul 15 20:03:26.629698 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.629658    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-nfqd2_a851c47d-8ee9-414c-bded-b74bb2105be5/kube-rbac-proxy/0.log"
Jul 15 20:03:26.659244 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.659190    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-nfg2x_da7a9830-a487-409e-9f40-0ae3607587b9/prometheus-operator-admission-webhook/0.log"
Jul 15 20:03:26.761895 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.761864    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-775c58594c-dv5vh_4c501cb9-c2a3-4ba1-a630-5d6e4b959a61/thanos-query/0.log"
Jul 15 20:03:26.784376 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.784341    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-775c58594c-dv5vh_4c501cb9-c2a3-4ba1-a630-5d6e4b959a61/kube-rbac-proxy-web/0.log"
Jul 15 20:03:26.811094 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.811065    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-775c58594c-dv5vh_4c501cb9-c2a3-4ba1-a630-5d6e4b959a61/kube-rbac-proxy/0.log"
Jul 15 20:03:26.834174 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.834144    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-775c58594c-dv5vh_4c501cb9-c2a3-4ba1-a630-5d6e4b959a61/prom-label-proxy/0.log"
Jul 15 20:03:26.861542 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.861495    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-775c58594c-dv5vh_4c501cb9-c2a3-4ba1-a630-5d6e4b959a61/kube-rbac-proxy-rules/0.log"
Jul 15 20:03:26.889062 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:26.889033    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-775c58594c-dv5vh_4c501cb9-c2a3-4ba1-a630-5d6e4b959a61/kube-rbac-proxy-metrics/0.log"
Jul 15 20:03:28.101757 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.101716    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"]
Jul 15 20:03:28.107338 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.107305    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.112865 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.112832    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"]
Jul 15 20:03:28.251185 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.251142    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0740412-c1ae-4205-a1b5-7eb604835ba0-sys\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.251353 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.251224    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e0740412-c1ae-4205-a1b5-7eb604835ba0-podres\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.251353 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.251274    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mcgq\" (UniqueName: \"kubernetes.io/projected/e0740412-c1ae-4205-a1b5-7eb604835ba0-kube-api-access-6mcgq\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.251353 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.251308    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e0740412-c1ae-4205-a1b5-7eb604835ba0-proc\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.251509 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.251404    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0740412-c1ae-4205-a1b5-7eb604835ba0-lib-modules\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.352918 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.352818    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0740412-c1ae-4205-a1b5-7eb604835ba0-sys\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.352918 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.352893    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e0740412-c1ae-4205-a1b5-7eb604835ba0-podres\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.353138 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.352938    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mcgq\" (UniqueName: \"kubernetes.io/projected/e0740412-c1ae-4205-a1b5-7eb604835ba0-kube-api-access-6mcgq\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.353138 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.352966    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e0740412-c1ae-4205-a1b5-7eb604835ba0-proc\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.353138 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.353045    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0740412-c1ae-4205-a1b5-7eb604835ba0-lib-modules\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.353300 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.353265    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0740412-c1ae-4205-a1b5-7eb604835ba0-lib-modules\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.353354 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.353328    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0740412-c1ae-4205-a1b5-7eb604835ba0-sys\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.353424 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.353401    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e0740412-c1ae-4205-a1b5-7eb604835ba0-podres\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.354326 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.353653    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e0740412-c1ae-4205-a1b5-7eb604835ba0-proc\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.363652 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.363621    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mcgq\" (UniqueName: \"kubernetes.io/projected/e0740412-c1ae-4205-a1b5-7eb604835ba0-kube-api-access-6mcgq\") pod \"perf-node-gather-daemonset-f9r8x\" (UID: \"e0740412-c1ae-4205-a1b5-7eb604835ba0\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.424241 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.424208    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:28.618148 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:28.617082    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"]
Jul 15 20:03:29.000522 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:29.000483    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x" event={"ID":"e0740412-c1ae-4205-a1b5-7eb604835ba0","Type":"ContainerStarted","Data":"86e472b32cb6f4ac51e8ee587f348e5d97cb911fa4658e7a4136171914785ef4"}
Jul 15 20:03:29.000719 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:29.000548    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x" event={"ID":"e0740412-c1ae-4205-a1b5-7eb604835ba0","Type":"ContainerStarted","Data":"4454327e2262a0191bea583161ec537c384a3bcf36035695fb8669fa9702ad07"}
Jul 15 20:03:29.001362 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:29.001337    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:29.017822 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:29.017769    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x" podStartSLOduration=1.017750117 podStartE2EDuration="1.017750117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:03:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:03:29.01762149 +0000 UTC m=+2881.087480891" watchObservedRunningTime="2026-07-15 20:03:29.017750117 +0000 UTC m=+2881.087609516"
Jul 15 20:03:29.208361 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:29.208335    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-67cc4c5686-jcz9v_3c680f8c-e12d-4c2f-9b93-ce802fd13841/console/0.log"
Jul 15 20:03:29.254133 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:29.254061    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-9z7pn_b8521d92-c8db-4396-9a67-ebb0c3acb319/download-server/0.log"
Jul 15 20:03:30.982139 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:30.982113    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pbzx7_eb78d556-5ce2-4076-835c-68be95f7b4e6/dns/0.log"
Jul 15 20:03:31.004535 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:31.004496    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pbzx7_eb78d556-5ce2-4076-835c-68be95f7b4e6/kube-rbac-proxy/0.log"
Jul 15 20:03:31.054437 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:31.054404    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q2249_19977ab6-3516-46be-931a-0dd0c38e808f/dns-node-resolver/0.log"
Jul 15 20:03:31.421589 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:31.421558    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:31.482030 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:31.481998    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l2wqc_05fa4c6c-3a40-411f-8125-d086982a264b/konnectivity-agent/0.log" limit=1024
Jul 15 20:03:31.674885 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:31.674783    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lgzhf_1f63fb98-ad9f-4986-af0f-714ae207a152/node-ca/0.log"
Jul 15 20:03:32.775420 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:32.775385    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59b767f7ff-8s2cv_136d57a8-1783-4d0d-9a15-77b54b17b3e2/kube-auth-proxy/0.log"
Jul 15 20:03:33.649627 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:33.649601    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hjh4v_b7cc6da0-11f7-4f0a-b5fe-2f5c90fce9c5/serve-healthcheck-canary/0.log"
Jul 15 20:03:34.194516 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:34.194490    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnzkb_c42c996f-f3ac-450e-bac3-79c20674a579/kube-rbac-proxy/0.log"
Jul 15 20:03:34.218528 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:34.218502    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnzkb_c42c996f-f3ac-450e-bac3-79c20674a579/exporter/0.log"
Jul 15 20:03:34.241818 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:34.241786    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnzkb_c42c996f-f3ac-450e-bac3-79c20674a579/extractor/0.log"
Jul 15 20:03:36.025625 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:36.025582    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-f9r8x"
Jul 15 20:03:37.058315 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:37.058285    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-s8vxc_acf759f2-50d0-45f3-a138-d8919ea53d58/manager/0.log"
Jul 15 20:03:37.274541 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:37.274516    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-78lfd_230e7abb-5a9a-422e-9302-d9d2091ee70a/server/0.log"
Jul 15 20:03:37.403473 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:37.403330    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mm4c8_5a570eb6-6ef3-49c2-9f97-620668fcef9e/manager/0.log"
Jul 15 20:03:37.433123 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:37.433094    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f46f5478c-m22fw_e411a261-a349-4b1c-9f8a-0269bef246b6/manager/0.log"
Jul 15 20:03:37.456467 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:37.456425    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f46f5478c-m22fw_e411a261-a349-4b1c-9f8a-0269bef246b6/copy-manifests/0.log"
Jul 15 20:03:38.764126 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:38.764095    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59b767f7ff-8s2cv_136d57a8-1783-4d0d-9a15-77b54b17b3e2/kube-auth-proxy/0.log"
Jul 15 20:03:38.903616 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:38.903589    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6c74497d4c-6fq5b_c5afab62-c61d-42c7-9a2c-ca77732acc8e/manager/0.log"
Jul 15 20:03:40.345877 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:40.345835    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59b767f7ff-8s2cv_136d57a8-1783-4d0d-9a15-77b54b17b3e2/kube-auth-proxy/0.log"
Jul 15 20:03:45.428025 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:45.427926    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2dpn6_209bcd45-469a-4a1d-9a46-3d512fcbd074/kube-multus/0.log"
Jul 15 20:03:45.622163 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:45.622134    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qm84p_586d100d-7063-44bf-9004-0265d19b5bc5/kube-multus-additional-cni-plugins/0.log"
Jul 15 20:03:45.646522 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:45.646493    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qm84p_586d100d-7063-44bf-9004-0265d19b5bc5/egress-router-binary-copy/0.log"
Jul 15 20:03:45.672766 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:45.672741    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qm84p_586d100d-7063-44bf-9004-0265d19b5bc5/cni-plugins/0.log"
Jul 15 20:03:45.697649 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:45.697589    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qm84p_586d100d-7063-44bf-9004-0265d19b5bc5/bond-cni-plugin/0.log"
Jul 15 20:03:45.721395 ip-10-0-143-114 kubenswrapper[2560]: I0715 20:03:45.721373    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qm84p_586d100d-7063-44bf-9004-0265d19b5bc5/routeoverride-cni/0.log"