Jul 10 03:30:35.053418 ip-10-0-143-142 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 10 03:30:35.053435 ip-10-0-143-142 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 10 03:30:35.053446 ip-10-0-143-142 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 10 03:30:35.053796 ip-10-0-143-142 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 10 03:30:45.247320 ip-10-0-143-142 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 10 03:30:45.247340 ip-10-0-143-142 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 6c4616f4e97f458d90a4af02a5d16d05 --
Jul 10 03:32:44.365774 ip-10-0-143-142 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.800609    2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805219    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805233    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805236    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805241    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805246    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805249    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805252    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 03:32:44.814782 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805255    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805258    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805261    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805264    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805266    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805269    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805272    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805275    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805277    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805280    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805283    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805287    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805289    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805293    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805296    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805299    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805301    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805304    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805307    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 03:32:44.815682 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805310    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805315    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805320    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805324    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805327    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805330    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805333    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805336    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805340    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805344    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805347    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805350    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805352    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805355    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805358    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805361    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805363    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805366    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805369    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 03:32:44.816568 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805372    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805374    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805378    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805381    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805384    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805387    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805390    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805392    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805395    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805398    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805400    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805403    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805406    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805409    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805413    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805416    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805419    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805422    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805425    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805429    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 03:32:44.817136 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805432    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805435    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805437    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805440    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805443    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805446    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805449    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805451    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805454    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805457    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805460    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805463    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805465    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805468    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805471    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805473    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805476    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805480    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805483    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805486    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 03:32:44.817752 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805488    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805491    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805494    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805496    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805500    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805503    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805508    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805512    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805516    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805519    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805522    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805525    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805529    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.805544    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807145    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807153    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807157    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807161    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807164    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 03:32:44.818305 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807168    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807171    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807174    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807177    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807180    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807183    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807186    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807189    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807191    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807194    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807197    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807200    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807203    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807206    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807208    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807211    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807215    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807217    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807220    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807223    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807226    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 03:32:44.818904 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807229    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807232    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807235    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807238    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807241    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807244    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807247    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807251    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807254    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807256    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807259    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807262    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807264    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807268    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807272    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807276    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807279    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807283    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807286    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 03:32:44.819526 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807289    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807293    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807296    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807299    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807302    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807306    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807309    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807311    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807314    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807317    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807320    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807322    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807325    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807328    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807330    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807333    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807337    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807340    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807344    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 03:32:44.820137 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807348    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807351    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807356    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807358    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807362    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807364    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807367    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807372    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807376    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807379    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807382    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807385    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807388    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807390    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807393    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807396    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807398    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807401    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807404    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 03:32:44.820819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807407    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807410    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807413    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807415    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807418    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807421    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807423    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807426    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807429    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807432    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807434    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807438    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807441    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807452    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807456    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807459    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807880    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807885    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807888    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807891    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 03:32:44.821476 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807894    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807897    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807900    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807903    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807906    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807909    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807912    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807914    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807917    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807920    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807923    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807932    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807935    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807938    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807941    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807944    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807947    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807950    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807952    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807955    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 03:32:44.822151 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807958    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807962    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807965    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807968    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807970    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807973    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807976    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807979    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807982    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807984    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807987    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807990    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807992    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807995    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.807998    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808002    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808004    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808007    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808009    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808013    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 03:32:44.822796 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808016    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808019    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808021    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808026    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808029    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808031    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808034    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808037    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808040    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808042    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808045    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808048    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808051    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808053    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808056    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808061    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808064    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808067    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808070    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 03:32:44.823463 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808074    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808077    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808079    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808082    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808085    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808087    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808090    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808093    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808096    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808100    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808103    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808107    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808111    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808113    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808116    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808119    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808123    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808126    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808128    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 03:32:44.824063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808131    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808134    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808137    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808139    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808142    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808146    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808149    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808151    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808154    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808157    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808159    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808163    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808167    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808170    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808173    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808176    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.808179    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808266    2571 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808277    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808287    2571 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 03:32:44.824654 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808294    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808300    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808303    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808308    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808312    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808316    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808319    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808323    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808326    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808331    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808335    2571 flags.go:64] FLAG: --cgroup-root=""
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808338    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808341    2571 flags.go:64] FLAG: --client-ca-file=""
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808344    2571 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808348    2571 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808352    2571 flags.go:64] FLAG: --cluster-domain=""
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808355    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808360    2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808363    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808366    2571 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808370    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808374    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808377    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 03:32:44.825266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808381    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808384    2571 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808387    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808390    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808394    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808398    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808405    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808409    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808412    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808415    2571 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808418    2571 flags.go:64] FLAG: --enable-server="true"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808421    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808427    2571 flags.go:64] FLAG: --event-burst="100"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808430    2571 flags.go:64] FLAG: --event-qps="50"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808433    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808436    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808440    2571 flags.go:64] FLAG: --eviction-hard=""
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808444    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808449    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808452    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808456    2571 flags.go:64] FLAG: --eviction-soft=""
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808459    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808462    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808465    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808468    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 03:32:44.826025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808471    2571 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808476    2571 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808479    2571 flags.go:64] FLAG: --feature-gates=""
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808484    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808487    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808490    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808494    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808497    2571 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808500    2571 flags.go:64] FLAG: --help="false"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808503    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-143-142.ec2.internal"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808507    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808510    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808514    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808518    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808522    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808525    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808528    2571 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808531    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808549    2571 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808553    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808556    2571 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808560    2571 flags.go:64] FLAG: --kube-reserved=""
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808563    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808566    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 03:32:44.826798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808569    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808574    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808577    2571 flags.go:64] FLAG: --lock-file=""
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808580    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808583    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808587    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808593    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808596    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808599    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808604    2571 flags.go:64] FLAG: --logging-format="text"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808607    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808610    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808614    2571 flags.go:64] FLAG: --manifest-url=""
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808617    2571 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808622    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808625    2571 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808629    2571 flags.go:64] FLAG: --max-pods="110"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808633    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808636    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808639    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808643    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808646    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808650    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808653    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808660    2571 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 03:32:44.827774 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808664    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808667    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808671    2571 flags.go:64] FLAG: --pod-cidr=""
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808674    2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808680    2571 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808683    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808686    2571 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808689    2571 flags.go:64] FLAG: --port="10250"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808693    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808697    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c009515f2c8b0c05"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808701    2571 flags.go:64] FLAG: --qos-reserved=""
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808704    2571 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808707    2571 flags.go:64] FLAG: --register-node="true"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808711    2571 flags.go:64] FLAG: --register-with-taints=""
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808715    2571 flags.go:64] FLAG: --registry-burst="10"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808718    2571 flags.go:64] FLAG: --registry-qps="5"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808722    2571 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808726    2571 flags.go:64] FLAG: --reserved-memory=""
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808729    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808733    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808736    2571 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808739    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808742    2571 flags.go:64] FLAG: --runonce="false"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808745    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808748    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 03:32:44.996588 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808752    2571 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 03:32:44.879163 ip-10-0-143-142 systemd[1]: Started Kubernetes Kubelet.
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808755    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808759    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808762    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808766    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808769    2571 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808772    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808776    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808779    2571 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808782    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808786    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808789    2571 flags.go:64] FLAG: --system-cgroups=""
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808792    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808797    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808800    2571 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808803    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808810    2571 flags.go:64] FLAG: --tls-min-version=""
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808813    2571 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808816    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808819    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808822    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808826    2571 flags.go:64] FLAG: --v="2"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808830    2571 flags.go:64] FLAG: --version="false"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808836    2571 flags.go:64] FLAG: --vmodule=""
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808841    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 03:32:45.028126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.808845    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809010    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809015    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809018    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809022    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809026    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809030    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809033    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809036    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809039    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809044    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809048    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809051    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809055    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809058    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809061    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809064    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809067    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809070    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 03:32:45.029319 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809073    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809077    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809081    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809085    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809089    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809092    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809095    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809098    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809102    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809104    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809107    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809112    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809115    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809118    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809122    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809126    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809129    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809132    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809135    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 03:32:45.030168 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809138    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809141    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809145    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809148    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809151    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809154    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809157    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809160    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809163    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809166    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809169    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809171    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809174    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809177    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809180    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809182    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809185    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809189    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809193    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 03:32:45.030866 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809196    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809199    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809202    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809204    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809207    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809211    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809214    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809217    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809219    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809222    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809225    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809228    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809230    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809233    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809236    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809239    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809241    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809244    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809247    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809250    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 03:32:45.031500 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809252    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809255    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809259    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809262    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809264    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809267    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809270    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809273    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809275    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809278    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809282    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809285    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809288    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809291    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809294    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809296    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809299    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809303    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809306    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809308    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 03:32:45.032223 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809311    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809315    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.809317    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.809322    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.817458    2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.817476    2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.817529    2571 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.817539    2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817606    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817613    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817617    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817621    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817624    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817629    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817632    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 03:32:45.032898 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817635    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817638    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817642    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817645    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817648    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817651    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817654    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817658    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817661    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817665    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817667    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817671    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817674    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817676    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817679    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817682    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817685    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817688    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817691    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817694    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 03:32:45.033454 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817696    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817701    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817706    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817710    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817713    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817715    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817718    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817721    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817724    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817727    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817730    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817733    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817736    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817738    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817742    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817745    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817748    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817751    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817754    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817758    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 03:32:45.034122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817761    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817764    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817767    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817770    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817773    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817776    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817779    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817782    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817784    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817787    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817790    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817793    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817796    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817799    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817802    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817805    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817808    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817811    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817814    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817817    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 03:32:45.034872 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817820    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817823    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817825    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817829    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817831    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817834    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817837    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817840    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817845    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817848    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817852    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817855    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817858    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817861    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817864    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817866    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817870    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817873    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817876    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 03:32:45.035510 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817879    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817882    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817886    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817891    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817895    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817898    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817902    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817905    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817907    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817910    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817915    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817918    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.817921    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.817927    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818053    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 03:32:45.036173 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818058    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818062    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818065    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818069    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818072    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818075    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818079    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818082    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818084    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818087    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818091    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818096    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818101    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818104    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818107    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818111    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818114    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818117    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818121    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818124    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 03:32:45.036650 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818127    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818131    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818134    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818137    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818141    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818145    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818148    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818151    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818153    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818156    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818159    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818161    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818164    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818167    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818170    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818172    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818175    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818178    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818181    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818186    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 03:32:45.037482 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818189    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818192    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818195    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818197    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818201    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818204    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818207    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818210    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818213    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818216    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818219    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818221    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818224    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818227    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818230    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818233    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818236    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818239    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818241    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818244    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 03:32:45.038161 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818247    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818250    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818253    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818256    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818259    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818262    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818264    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818267    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818270    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818273    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818276    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818280    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818283    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818286    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818288    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818291    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818294    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818297    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818300    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818303    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 03:32:45.038832 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818306    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818309    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818313    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818317    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818320    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818323    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818328    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818331    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818334    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818337    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818340    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818343    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818345    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818348    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818351    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818354    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818357    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 03:32:45.039465 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:44.818360    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 03:32:45.040085 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.818365    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 03:32:45.040085 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.818491    2571 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 03:32:45.040085 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.820605    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 03:32:45.040085 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.822011    2571 server.go:1019] "Starting client certificate rotation"
Jul 10 03:32:45.040085 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.823718    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 03:32:45.040085 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.823763    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 03:32:45.040085 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.824283    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 03:32:45.040085 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.826740    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 03:32:45.040085 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.838442    2571 log.go:25] "Validated CRI v1 runtime API"
Jul 10 03:32:45.040085 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.841187    2571 log.go:25] "Validated CRI v1 image API"
Jul 10 03:32:45.040085 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.842211    2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 03:32:45.040085 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.847436    2571 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ce77b41b-e69d-4727-aa30-8ea9141195c3:/dev/nvme0n1p3 dc847df5-0d10-4f1d-9a26-c006e391863f:/dev/nvme0n1p4]
Jul 10 03:32:45.040460 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.847453    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 03:32:45.040591 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.855573    2571 manager.go:216] Machine: {Timestamp:2026-07-10 03:32:44.854172205 +0000 UTC m=+0.368907502 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100553 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec255aa3a2227f82a420c678cb55b79d SystemUUID:ec255aa3-a222-7f82-a420-c678cb55b79d BootID:6c4616f4-e97f-458d-90a4-af02a5d16d05 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ef:d8:d3:a8:7b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ef:d8:d3:a8:7b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:2d:f3:41:c3:8b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 03:32:45.040591 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.855689    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 03:32:45.040591 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.855781    2571 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 03:32:45.040591 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.856296    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 03:32:45.040591 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.858741    2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 03:32:45.040591 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.858769    2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-142.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 03:32:45.040591 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.860024    2571 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.860035    2571 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.860044    2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.861505    2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.862670    2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.866495    2571 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.866646    2571 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.866675    2571 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.866692    2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.866702    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.866713    2571 kubelet.go:398] "Adding apiserver pod source"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.866723    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.867070    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.867090    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.869866    2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.871630    2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.871661    2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.871831    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.871849    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.871864    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.871878    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.871896    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.871909    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.872387    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.872400    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.872411    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.872420    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.872433    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.872446    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.873328    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.873340    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.877039    2571 server.go:1268] "Started kubelet"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.877184    2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.877315    2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.877431    2571 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.877893    2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-142.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:44.877943    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:44.878034    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-142.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.878808    2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.881924    2571 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 03:32:45.041150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.884541    2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.884563    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:44.884653    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:44.885172    2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.885230    2571 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.885245    2571 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.885458    2571 factory.go:55] Registering systemd factory
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.885471    2571 factory.go:223] Registration of the systemd container factory successfully
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.888795    2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.890468    2571 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.890502    2571 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.890737    2571 factory.go:153] Registering CRI-O factory
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.890756    2571 factory.go:223] Registration of the crio container factory successfully
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.890829    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.890854    2571 factory.go:103] Registering Raw factory
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.890871    2571 manager.go:1196] Started watching for new ooms in manager
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:44.890484    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-142.ec2.internal.18c0d031d491bc8d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-142.ec2.internal,UID:ip-10-0-143-142.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-142.ec2.internal,},FirstTimestamp:2026-07-10 03:32:44.877012109 +0000 UTC m=+0.391747406,LastTimestamp:2026-07-10 03:32:44.877012109 +0000 UTC m=+0.391747406,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-142.ec2.internal,}"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.894486    2571 manager.go:349] Starting recovery of all containers
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:44.898837    2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-142.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:44.898996    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.900838    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gn7lx"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.905979    2571 manager.go:354] Recovery completed
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.910359    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.914004    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gn7lx"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.914598    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.914738    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.914749    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.915347    2571 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.915359    2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.915361    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.915374    2571 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.916452    2571 policy_none.go:49] "None policy: Start"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.916512    2571 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 03:32:45.042395 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.916531    2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.916775    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.916807    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.916823    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:44.916767    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-142.ec2.internal.18c0d031d6d11ef3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-142.ec2.internal,UID:ip-10-0-143-142.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-142.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-142.ec2.internal,},FirstTimestamp:2026-07-10 03:32:44.914720499 +0000 UTC m=+0.429455799,LastTimestamp:2026-07-10 03:32:44.914720499 +0000 UTC m=+0.429455799,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-142.ec2.internal,}"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.917581    2571 policy_none.go:47] "Start"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.968116    2571 manager.go:337] "Starting Device Plugin manager"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:44.968160    2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.968174    2571 server.go:87] "Starting device plugin registration server"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.968450    2571 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.968462    2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.968561    2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.968649    2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:44.968657    2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:44.969353    2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:44.969401    2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.071047    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.072279    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.072318    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.072331    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.072360    2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.080969    2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:45.080990    2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-142.ec2.internal\": node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:45.097590    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.118263    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.119552    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.119589    2571 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.119610    2571 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:45.119648    2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.122486    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:45.197881    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:45.286681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.220724    2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal"]
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.220824    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.224063    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.224093    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.224103    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.226564    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.226693    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.226720    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.227725    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.227735    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.227751    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.227756    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.227762    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.227766    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.230099    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.230125    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.230882    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.230907    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.230923    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeHasSufficientPID"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:45.247124    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-142.ec2.internal\" not found" node="ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.288174 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:45.254752    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-142.ec2.internal\" not found" node="ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.293047    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/343c2c99cf08d1b78e530233805856b4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal\" (UID: \"343c2c99cf08d1b78e530233805856b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.293088    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0b3b8930aa1846297e408b672c61a6ab-config\") pod \"kube-apiserver-proxy-ip-10-0-143-142.ec2.internal\" (UID: \"0b3b8930aa1846297e408b672c61a6ab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.293114    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/343c2c99cf08d1b78e530233805856b4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal\" (UID: \"343c2c99cf08d1b78e530233805856b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:45.298949    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.393971    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/343c2c99cf08d1b78e530233805856b4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal\" (UID: \"343c2c99cf08d1b78e530233805856b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.394027    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/343c2c99cf08d1b78e530233805856b4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal\" (UID: \"343c2c99cf08d1b78e530233805856b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.394055    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0b3b8930aa1846297e408b672c61a6ab-config\") pod \"kube-apiserver-proxy-ip-10-0-143-142.ec2.internal\" (UID: \"0b3b8930aa1846297e408b672c61a6ab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.394096    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0b3b8930aa1846297e408b672c61a6ab-config\") pod \"kube-apiserver-proxy-ip-10-0-143-142.ec2.internal\" (UID: \"0b3b8930aa1846297e408b672c61a6ab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.394153    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/343c2c99cf08d1b78e530233805856b4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal\" (UID: \"343c2c99cf08d1b78e530233805856b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.394180    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/343c2c99cf08d1b78e530233805856b4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal\" (UID: \"343c2c99cf08d1b78e530233805856b4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:45.399407    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:45.500308    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.552442    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.582156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.557482    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:45.874758 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:45.601407    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:45.874758 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:45.701718    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:45.874758 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:45.802066    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:45.874758 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.821883    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 03:32:45.874758 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.822115    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 03:32:45.979440 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:45.902510    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:45.979440 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.917766    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 03:27:45 +0000 UTC" deadline="2028-02-02 13:42:47.294299908 +0000 UTC"
Jul 10 03:32:45.979440 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:45.917814    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13738h10m1.376490119s"
Jul 10 03:32:46.177741 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:46.003311    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:46.177741 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:46.104505    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:46.297122 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:46.205159    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:46.316308 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:46.306095    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:46.422837 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:46.406582    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:46.546753 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:46.507648    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:46.667011 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:46.608399    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:46.775339 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:46.709286    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:46.810228 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:46.810187    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-142.ec2.internal\" not found"
Jul 10 03:32:46.861617 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:46.861571    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b3b8930aa1846297e408b672c61a6ab.slice/crio-774208353bfb64198b111d30c422b1597f5d96bfd718b38d5de6f841fc56909d WatchSource:0}: Error finding container 774208353bfb64198b111d30c422b1597f5d96bfd718b38d5de6f841fc56909d: Status 404 returned error can't find the container with id 774208353bfb64198b111d30c422b1597f5d96bfd718b38d5de6f841fc56909d
Jul 10 03:32:46.868475 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:46.868441    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 03:32:46.873540 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:46.873204    2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 03:32:46.873540 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:46.873253    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 03:32:46.874291 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:46.874264    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod343c2c99cf08d1b78e530233805856b4.slice/crio-433d821326c1d7c4320bd908fcd394f627d36bad76a3dc7f22bc7ec805212693 WatchSource:0}: Error finding container 433d821326c1d7c4320bd908fcd394f627d36bad76a3dc7f22bc7ec805212693: Status 404 returned error can't find the container with id 433d821326c1d7c4320bd908fcd394f627d36bad76a3dc7f22bc7ec805212693
Jul 10 03:32:46.875762 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:46.875736    2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 03:32:46.885784 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:46.885758    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:46.885876 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:46.885781    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 03:32:46.896276 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:46.896250    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 03:32:46.897430 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:46.897413    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal"
Jul 10 03:32:46.897782 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:46.897765    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 03:32:46.905548 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:46.905517    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 03:32:46.916012 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:46.915991    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wkwxr"
Jul 10 03:32:46.923644 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:46.923622    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wkwxr"
Jul 10 03:32:47.124677 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.124554    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" event={"ID":"343c2c99cf08d1b78e530233805856b4","Type":"ContainerStarted","Data":"433d821326c1d7c4320bd908fcd394f627d36bad76a3dc7f22bc7ec805212693"}
Jul 10 03:32:47.125630 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.125604    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal" event={"ID":"0b3b8930aa1846297e408b672c61a6ab","Type":"ContainerStarted","Data":"774208353bfb64198b111d30c422b1597f5d96bfd718b38d5de6f841fc56909d"}
Jul 10 03:32:47.416199 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.416113    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 03:32:47.869997 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.869922    2571 apiserver.go:52] "Watching apiserver"
Jul 10 03:32:47.875972 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.875940    2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 03:32:47.876480 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.876428    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-8458m","kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt","openshift-cluster-node-tuning-operator/tuned-h6nnw","openshift-dns/node-resolver-9p7cz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal","openshift-multus/multus-7dqr4","openshift-multus/network-metrics-daemon-j6vpn","kube-system/global-pull-secret-syncer-vlr5v","openshift-image-registry/node-ca-xnfdx","openshift-multus/multus-additional-cni-plugins-dhvb5","openshift-network-diagnostics/network-check-target-zkhkn","openshift-network-operator/iptables-alerter-pmhgt","openshift-ovn-kubernetes/ovnkube-node-gxvf2"]
Jul 10 03:32:47.879524 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.879401    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.881853 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.881652    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 03:32:47.881853 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.881710    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 03:32:47.881853 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.881776    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 03:32:47.882088 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.882014    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pmhgt"
Jul 10 03:32:47.882390 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.882370    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 03:32:47.882681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.882513    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-v4f6v\""
Jul 10 03:32:47.883863 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.883844    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 03:32:47.884502 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.884285    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 03:32:47.884502 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.884299    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 03:32:47.884654 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.884601    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z6qcx\""
Jul 10 03:32:47.884654 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.884643    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:47.886872 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.886828    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 03:32:47.887035 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.887016    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 03:32:47.887352 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.887332    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8t6g8\""
Jul 10 03:32:47.887435 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.887369    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 03:32:47.887741 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.887720    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.889854 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.889831    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gqct6\""
Jul 10 03:32:47.889953 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.889885    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 03:32:47.890061 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.890030    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 03:32:47.892291 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.892232    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9p7cz"
Jul 10 03:32:47.895203 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.895185    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 03:32:47.895639 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.895491    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ldt9j\""
Jul 10 03:32:47.895639 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.895533    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 03:32:47.895900 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.895879    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:47.898169 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.898150    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 03:32:47.898730 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.898554    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 03:32:47.898730 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.898679    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 03:32:47.898953 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.898931    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:47.899068 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:47.899033    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:32:47.899253 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.899148    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2x47q\""
Jul 10 03:32:47.899351 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.899265    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 03:32:47.899639 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.899619    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 03:32:47.899896 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.899876    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 03:32:47.901646 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.901610    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8458m"
Jul 10 03:32:47.903971 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.903950    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 03:32:47.904124 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.904103    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qnp25\""
Jul 10 03:32:47.904204 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.904105    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 03:32:47.904263 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.904111    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:47.904336 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:47.904308    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:32:47.907181 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.907145    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xnfdx"
Jul 10 03:32:47.908615 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.908592    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-os-release\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.908713 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.908632    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztlwz\" (UniqueName: \"kubernetes.io/projected/ebc252f0-1441-4bf2-a593-8eabd8aba528-kube-api-access-ztlwz\") pod \"iptables-alerter-pmhgt\" (UID: \"ebc252f0-1441-4bf2-a593-8eabd8aba528\") " pod="openshift-network-operator/iptables-alerter-pmhgt"
Jul 10 03:32:47.908713 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.908673    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-socket-dir\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:47.908713 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.908701    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-systemd\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.908835 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.908738    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-run\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.908835 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.908763    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b1d7593-1d0c-4166-8dc8-733dc6c4f55c-tmp-dir\") pod \"node-resolver-9p7cz\" (UID: \"4b1d7593-1d0c-4166-8dc8-733dc6c4f55c\") " pod="openshift-dns/node-resolver-9p7cz"
Jul 10 03:32:47.908835 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.908789    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kvhn\" (UniqueName: \"kubernetes.io/projected/4b1d7593-1d0c-4166-8dc8-733dc6c4f55c-kube-api-access-5kvhn\") pod \"node-resolver-9p7cz\" (UID: \"4b1d7593-1d0c-4166-8dc8-733dc6c4f55c\") " pod="openshift-dns/node-resolver-9p7cz"
Jul 10 03:32:47.908938 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.908835    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ebc252f0-1441-4bf2-a593-8eabd8aba528-host-slash\") pod \"iptables-alerter-pmhgt\" (UID: \"ebc252f0-1441-4bf2-a593-8eabd8aba528\") " pod="openshift-network-operator/iptables-alerter-pmhgt"
Jul 10 03:32:47.908938 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.908862    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:47.908938 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.908890    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-kubernetes\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.909093 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.908939    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-sysctl-conf\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.909093 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.908976    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-host\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.909093 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909005    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-system-cni-dir\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.909093 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909040    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-run-netns\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.909093 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909067    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shk2n\" (UniqueName: \"kubernetes.io/projected/638ec913-d0a6-4937-bc25-adc4d6fc5897-kube-api-access-shk2n\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.909340 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909092    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwvpz\" (UniqueName: \"kubernetes.io/projected/baa2bc4a-1a56-4648-a3ee-703e4258204b-kube-api-access-hwvpz\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:47.909340 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909100    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 03:32:47.909340 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909116    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-sys\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.909340 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909141    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c014e314-0ed8-4ecc-8c60-5447472f4121-tmp\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.909340 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909184    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/638ec913-d0a6-4937-bc25-adc4d6fc5897-cni-binary-copy\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.909340 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909209    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-device-dir\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:47.909340 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909251    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-lib-modules\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.909340 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909278    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-var-lib-kubelet\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.909340 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909333    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 03:32:47.909809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909337    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-run-k8s-cni-cncf-io\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.909809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909381    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-hostroot\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.909809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909411    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 03:32:47.909809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909426    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ebc252f0-1441-4bf2-a593-8eabd8aba528-iptables-alerter-script\") pod \"iptables-alerter-pmhgt\" (UID: \"ebc252f0-1441-4bf2-a593-8eabd8aba528\") " pod="openshift-network-operator/iptables-alerter-pmhgt"
Jul 10 03:32:47.909809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909536    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-sys-fs\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:47.909809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909570    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4b1d7593-1d0c-4166-8dc8-733dc6c4f55c-hosts-file\") pod \"node-resolver-9p7cz\" (UID: \"4b1d7593-1d0c-4166-8dc8-733dc6c4f55c\") " pod="openshift-dns/node-resolver-9p7cz"
Jul 10 03:32:47.909809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909578    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cbf46\""
Jul 10 03:32:47.909809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909598    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-cnibin\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.909809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909670    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-var-lib-cni-multus\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.909809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909702    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/638ec913-d0a6-4937-bc25-adc4d6fc5897-multus-daemon-config\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.909809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909735    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-run-multus-certs\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.909809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909777    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-etc-kubernetes\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.909809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909810    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-registration-dir\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:47.910419 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909836    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-sysconfig\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.910419 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909873    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-tuned\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.910419 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909930    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-multus-socket-dir-parent\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.910419 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.909962    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-var-lib-cni-bin\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.910419 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.910005    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-var-lib-kubelet\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.910419 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.910013    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:47.910419 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.910030    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-multus-conf-dir\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.910419 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.910056    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-etc-selinux\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:47.910419 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.910098    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-modprobe-d\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.910419 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.910122    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-sysctl-d\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.910419 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.910146    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chtsw\" (UniqueName: \"kubernetes.io/projected/c014e314-0ed8-4ecc-8c60-5447472f4121-kube-api-access-chtsw\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:47.910419 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.910187    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-multus-cni-dir\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:47.911876 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.911855    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 03:32:47.912403 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.912172    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qnc2d\""
Jul 10 03:32:47.912403 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.912210    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 03:32:47.912403 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.912365    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:32:47.912642 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:47.912438    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:32:47.927418 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.927344    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 03:27:47 +0000 UTC" deadline="2028-01-12 18:04:07.828940588 +0000 UTC"
Jul 10 03:32:47.927418 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.927375    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13238h31m19.90156957s"
Jul 10 03:32:47.990339 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:47.990309    2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 03:32:48.010529 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.010488    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/26fcd562-4c93-4255-9a6d-0c4a362b4b72-os-release\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.010714 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.010543    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-run\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.010714 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.010606    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kvhn\" (UniqueName: \"kubernetes.io/projected/4b1d7593-1d0c-4166-8dc8-733dc6c4f55c-kube-api-access-5kvhn\") pod \"node-resolver-9p7cz\" (UID: \"4b1d7593-1d0c-4166-8dc8-733dc6c4f55c\") " pod="openshift-dns/node-resolver-9p7cz"
Jul 10 03:32:48.010714 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.010670    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-node-log\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.010714 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.010686    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-run\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.010714 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.010710    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/26fcd562-4c93-4255-9a6d-0c4a362b4b72-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.010995 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.010744    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d4200887-8985-4033-8fb6-5660c89ff1f7-ovnkube-script-lib\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.010995 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.010775    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/26fcd562-4c93-4255-9a6d-0c4a362b4b72-cni-binary-copy\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.010995 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.010802    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/26fcd562-4c93-4255-9a6d-0c4a362b4b72-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.010995 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.010844    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-run-netns\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.010995 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.010877    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shk2n\" (UniqueName: \"kubernetes.io/projected/638ec913-d0a6-4937-bc25-adc4d6fc5897-kube-api-access-shk2n\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.010995 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.010916    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwvpz\" (UniqueName: \"kubernetes.io/projected/baa2bc4a-1a56-4648-a3ee-703e4258204b-kube-api-access-hwvpz\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.010995 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.010925    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-run-netns\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.011344 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011022    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-var-lib-openvswitch\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.011344 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011078    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-656sj\" (UniqueName: \"kubernetes.io/projected/26fcd562-4c93-4255-9a6d-0c4a362b4b72-kube-api-access-656sj\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.011344 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011118    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-device-dir\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.011344 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011188    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-lib-modules\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.011344 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011214    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-var-lib-kubelet\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.011344 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011233    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-device-dir\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.011344 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011245    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d4200887-8985-4033-8fb6-5660c89ff1f7-env-overrides\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.011344 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011280    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8c21afc9-23db-4b36-b482-8c244a254df6-agent-certs\") pod \"konnectivity-agent-8458m\" (UID: \"8c21afc9-23db-4b36-b482-8c244a254df6\") " pod="kube-system/konnectivity-agent-8458m"
Jul 10 03:32:48.011344 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011304    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-hostroot\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.011344 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011324    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-var-lib-kubelet\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011385    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-lib-modules\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011391    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-hostroot\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011327    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ebc252f0-1441-4bf2-a593-8eabd8aba528-iptables-alerter-script\") pod \"iptables-alerter-pmhgt\" (UID: \"ebc252f0-1441-4bf2-a593-8eabd8aba528\") " pod="openshift-network-operator/iptables-alerter-pmhgt"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011431    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-sys-fs\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011449    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/859d3328-63da-49c4-a871-2b45a4960764-kubelet-config\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011492    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sghc\" (UniqueName: \"kubernetes.io/projected/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-kube-api-access-2sghc\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011512    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-sys-fs\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011519    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/26fcd562-4c93-4255-9a6d-0c4a362b4b72-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011547    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-var-lib-cni-multus\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011570    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-etc-kubernetes\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011586    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-sysconfig\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011601    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-tuned\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011622    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b1d7593-1d0c-4166-8dc8-733dc6c4f55c-tmp-dir\") pod \"node-resolver-9p7cz\" (UID: \"4b1d7593-1d0c-4166-8dc8-733dc6c4f55c\") " pod="openshift-dns/node-resolver-9p7cz"
Jul 10 03:32:48.011843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011649    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-systemd-units\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011655    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-etc-kubernetes\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011683    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-var-lib-kubelet\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011713    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-sysconfig\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011709    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-etc-selinux\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011743    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-modprobe-d\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011769    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-run-systemd\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011793    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfe8cc34-69bf-4d9e-a945-b1c867b6111a-host\") pod \"node-ca-xnfdx\" (UID: \"bfe8cc34-69bf-4d9e-a945-b1c867b6111a\") " pod="openshift-image-registry/node-ca-xnfdx"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011820    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/26fcd562-4c93-4255-9a6d-0c4a362b4b72-cnibin\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011844    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztlwz\" (UniqueName: \"kubernetes.io/projected/ebc252f0-1441-4bf2-a593-8eabd8aba528-kube-api-access-ztlwz\") pod \"iptables-alerter-pmhgt\" (UID: \"ebc252f0-1441-4bf2-a593-8eabd8aba528\") " pod="openshift-network-operator/iptables-alerter-pmhgt"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011855    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-var-lib-cni-multus\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011865    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-var-lib-kubelet\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011918    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-systemd\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.011974    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-modprobe-d\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.012500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012003    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ebc252f0-1441-4bf2-a593-8eabd8aba528-iptables-alerter-script\") pod \"iptables-alerter-pmhgt\" (UID: \"ebc252f0-1441-4bf2-a593-8eabd8aba528\") " pod="openshift-network-operator/iptables-alerter-pmhgt"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012024    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-etc-openvswitch\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012054    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-etc-selinux\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012069    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b1d7593-1d0c-4166-8dc8-733dc6c4f55c-tmp-dir\") pod \"node-resolver-9p7cz\" (UID: \"4b1d7593-1d0c-4166-8dc8-733dc6c4f55c\") " pod="openshift-dns/node-resolver-9p7cz"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012082    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-systemd\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012083    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-cni-bin\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012128    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bfe8cc34-69bf-4d9e-a945-b1c867b6111a-serviceca\") pod \"node-ca-xnfdx\" (UID: \"bfe8cc34-69bf-4d9e-a945-b1c867b6111a\") " pod="openshift-image-registry/node-ca-xnfdx"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012165    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012198    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ebc252f0-1441-4bf2-a593-8eabd8aba528-host-slash\") pod \"iptables-alerter-pmhgt\" (UID: \"ebc252f0-1441-4bf2-a593-8eabd8aba528\") " pod="openshift-network-operator/iptables-alerter-pmhgt"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012209    2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012219    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012270    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012292    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ebc252f0-1441-4bf2-a593-8eabd8aba528-host-slash\") pod \"iptables-alerter-pmhgt\" (UID: \"ebc252f0-1441-4bf2-a593-8eabd8aba528\") " pod="openshift-network-operator/iptables-alerter-pmhgt"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012331    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-kubernetes\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012365    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-sysctl-conf\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.013130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012389    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-host\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012405    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-kubernetes\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012445    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d4200887-8985-4033-8fb6-5660c89ff1f7-ovnkube-config\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012489    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4200887-8985-4033-8fb6-5660c89ff1f7-ovn-node-metrics-cert\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012509    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-host\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012541    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-system-cni-dir\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012551    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-sysctl-conf\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012566    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-sys\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012620    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-system-cni-dir\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012626    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-sys\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012678    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c014e314-0ed8-4ecc-8c60-5447472f4121-tmp\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012701    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-kubelet\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012717    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-slash\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012739    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zwf6\" (UniqueName: \"kubernetes.io/projected/d4200887-8985-4033-8fb6-5660c89ff1f7-kube-api-access-6zwf6\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.013945 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012772    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z598\" (UniqueName: \"kubernetes.io/projected/bfe8cc34-69bf-4d9e-a945-b1c867b6111a-kube-api-access-9z598\") pod \"node-ca-xnfdx\" (UID: \"bfe8cc34-69bf-4d9e-a945-b1c867b6111a\") " pod="openshift-image-registry/node-ca-xnfdx"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012798    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8c21afc9-23db-4b36-b482-8c244a254df6-konnectivity-ca\") pod \"konnectivity-agent-8458m\" (UID: \"8c21afc9-23db-4b36-b482-8c244a254df6\") " pod="kube-system/konnectivity-agent-8458m"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012825    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/638ec913-d0a6-4937-bc25-adc4d6fc5897-cni-binary-copy\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012868    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzwqc\" (UniqueName: \"kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc\") pod \"network-check-target-zkhkn\" (UID: \"ef22232c-35eb-4554-8518-ea64524eefa1\") " pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012904    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-run-netns\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012930    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-run-openvswitch\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012958    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-run-k8s-cni-cncf-io\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012975    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4b1d7593-1d0c-4166-8dc8-733dc6c4f55c-hosts-file\") pod \"node-resolver-9p7cz\" (UID: \"4b1d7593-1d0c-4166-8dc8-733dc6c4f55c\") " pod="openshift-dns/node-resolver-9p7cz"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.012994    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-run-ovn\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013019    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-cnibin\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013042    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-run-k8s-cni-cncf-io\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013044    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/638ec913-d0a6-4937-bc25-adc4d6fc5897-multus-daemon-config\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013085    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-run-multus-certs\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013108    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-registration-dir\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.014688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013136    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-cni-netd\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013162    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/859d3328-63da-49c4-a871-2b45a4960764-dbus\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013210    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013237    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-multus-socket-dir-parent\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013280    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-var-lib-cni-bin\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013307    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-multus-conf-dir\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013325    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/638ec913-d0a6-4937-bc25-adc4d6fc5897-cni-binary-copy\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013331    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-sysctl-d\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013385    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chtsw\" (UniqueName: \"kubernetes.io/projected/c014e314-0ed8-4ecc-8c60-5447472f4121-kube-api-access-chtsw\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013416    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-log-socket\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013436    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-sysctl-d\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013444    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013473    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26fcd562-4c93-4255-9a6d-0c4a362b4b72-system-cni-dir\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013485    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-run-multus-certs\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.015396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013499    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-multus-cni-dir\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013543    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-os-release\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013551    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/638ec913-d0a6-4937-bc25-adc4d6fc5897-multus-daemon-config\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013553    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-registration-dir\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013569    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-socket-dir\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013649    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-multus-socket-dir-parent\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013788    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4b1d7593-1d0c-4166-8dc8-733dc6c4f55c-hosts-file\") pod \"node-resolver-9p7cz\" (UID: \"4b1d7593-1d0c-4166-8dc8-733dc6c4f55c\") " pod="openshift-dns/node-resolver-9p7cz"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013899    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-multus-conf-dir\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.013975    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-os-release\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.014008    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-multus-cni-dir\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.014014    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-host-var-lib-cni-bin\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.014011    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/baa2bc4a-1a56-4648-a3ee-703e4258204b-socket-dir\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.014048    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-run-ovn-kubernetes\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.014054    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/638ec913-d0a6-4937-bc25-adc4d6fc5897-cnibin\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.015935    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c014e314-0ed8-4ecc-8c60-5447472f4121-etc-tuned\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.016007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.016001    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c014e314-0ed8-4ecc-8c60-5447472f4121-tmp\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.019838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.019813    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kvhn\" (UniqueName: \"kubernetes.io/projected/4b1d7593-1d0c-4166-8dc8-733dc6c4f55c-kube-api-access-5kvhn\") pod \"node-resolver-9p7cz\" (UID: \"4b1d7593-1d0c-4166-8dc8-733dc6c4f55c\") " pod="openshift-dns/node-resolver-9p7cz"
Jul 10 03:32:48.020093 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.020070    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwvpz\" (UniqueName: \"kubernetes.io/projected/baa2bc4a-1a56-4648-a3ee-703e4258204b-kube-api-access-hwvpz\") pod \"aws-ebs-csi-driver-node-zdntt\" (UID: \"baa2bc4a-1a56-4648-a3ee-703e4258204b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.020093 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.020087    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shk2n\" (UniqueName: \"kubernetes.io/projected/638ec913-d0a6-4937-bc25-adc4d6fc5897-kube-api-access-shk2n\") pod \"multus-7dqr4\" (UID: \"638ec913-d0a6-4937-bc25-adc4d6fc5897\") " pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.023798 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.023778    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztlwz\" (UniqueName: \"kubernetes.io/projected/ebc252f0-1441-4bf2-a593-8eabd8aba528-kube-api-access-ztlwz\") pod \"iptables-alerter-pmhgt\" (UID: \"ebc252f0-1441-4bf2-a593-8eabd8aba528\") " pod="openshift-network-operator/iptables-alerter-pmhgt"
Jul 10 03:32:48.032946 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.032911    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chtsw\" (UniqueName: \"kubernetes.io/projected/c014e314-0ed8-4ecc-8c60-5447472f4121-kube-api-access-chtsw\") pod \"tuned-h6nnw\" (UID: \"c014e314-0ed8-4ecc-8c60-5447472f4121\") " pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.114514 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114455    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-cni-bin\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.114514 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114517    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bfe8cc34-69bf-4d9e-a945-b1c867b6111a-serviceca\") pod \"node-ca-xnfdx\" (UID: \"bfe8cc34-69bf-4d9e-a945-b1c867b6111a\") " pod="openshift-image-registry/node-ca-xnfdx"
Jul 10 03:32:48.114753 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114547    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:48.114753 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114579    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d4200887-8985-4033-8fb6-5660c89ff1f7-ovnkube-config\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.114753 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114607    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4200887-8985-4033-8fb6-5660c89ff1f7-ovn-node-metrics-cert\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.114753 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114635    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-kubelet\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.114753 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114660    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-slash\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.114753 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114687    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zwf6\" (UniqueName: \"kubernetes.io/projected/d4200887-8985-4033-8fb6-5660c89ff1f7-kube-api-access-6zwf6\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.114753 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114713    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z598\" (UniqueName: \"kubernetes.io/projected/bfe8cc34-69bf-4d9e-a945-b1c867b6111a-kube-api-access-9z598\") pod \"node-ca-xnfdx\" (UID: \"bfe8cc34-69bf-4d9e-a945-b1c867b6111a\") " pod="openshift-image-registry/node-ca-xnfdx"
Jul 10 03:32:48.114753 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114736    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8c21afc9-23db-4b36-b482-8c244a254df6-konnectivity-ca\") pod \"konnectivity-agent-8458m\" (UID: \"8c21afc9-23db-4b36-b482-8c244a254df6\") " pod="kube-system/konnectivity-agent-8458m"
Jul 10 03:32:48.115071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114761    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzwqc\" (UniqueName: \"kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc\") pod \"network-check-target-zkhkn\" (UID: \"ef22232c-35eb-4554-8518-ea64524eefa1\") " pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:32:48.115071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114784    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-run-netns\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.115071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114808    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-run-openvswitch\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.115071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114834    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-run-ovn\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.115071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114862    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-cni-netd\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.115071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114887    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/859d3328-63da-49c4-a871-2b45a4960764-dbus\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:48.115071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114920    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:48.115071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114951    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-log-socket\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.115071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.114977    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.115071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115015    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26fcd562-4c93-4255-9a6d-0c4a362b4b72-system-cni-dir\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.115071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115045    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-run-ovn-kubernetes\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.115071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115072    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/26fcd562-4c93-4255-9a6d-0c4a362b4b72-os-release\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115102    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-node-log\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115128    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/26fcd562-4c93-4255-9a6d-0c4a362b4b72-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115151    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d4200887-8985-4033-8fb6-5660c89ff1f7-ovnkube-script-lib\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115194    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/26fcd562-4c93-4255-9a6d-0c4a362b4b72-cni-binary-copy\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115211    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-cni-bin\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115217    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/26fcd562-4c93-4255-9a6d-0c4a362b4b72-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115283    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-var-lib-openvswitch\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115313    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-656sj\" (UniqueName: \"kubernetes.io/projected/26fcd562-4c93-4255-9a6d-0c4a362b4b72-kube-api-access-656sj\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115349    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d4200887-8985-4033-8fb6-5660c89ff1f7-env-overrides\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115376    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8c21afc9-23db-4b36-b482-8c244a254df6-agent-certs\") pod \"konnectivity-agent-8458m\" (UID: \"8c21afc9-23db-4b36-b482-8c244a254df6\") " pod="kube-system/konnectivity-agent-8458m"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115408    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/859d3328-63da-49c4-a871-2b45a4960764-kubelet-config\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115452    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sghc\" (UniqueName: \"kubernetes.io/projected/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-kube-api-access-2sghc\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115501    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/26fcd562-4c93-4255-9a6d-0c4a362b4b72-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.116311 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115541    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-systemd-units\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115575    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-run-systemd\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115599    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfe8cc34-69bf-4d9e-a945-b1c867b6111a-host\") pod \"node-ca-xnfdx\" (UID: \"bfe8cc34-69bf-4d9e-a945-b1c867b6111a\") " pod="openshift-image-registry/node-ca-xnfdx"
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115625    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/26fcd562-4c93-4255-9a6d-0c4a362b4b72-cnibin\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115655    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-etc-openvswitch\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.115684    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115739    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-etc-openvswitch\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.115780    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret podName:859d3328-63da-49c4-a871-2b45a4960764 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:48.61575735 +0000 UTC m=+4.130492658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret") pod "global-pull-secret-syncer-vlr5v" (UID: "859d3328-63da-49c4-a871-2b45a4960764") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115874    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-kubelet\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115928    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26fcd562-4c93-4255-9a6d-0c4a362b4b72-system-cni-dir\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115965    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/26fcd562-4c93-4255-9a6d-0c4a362b4b72-os-release\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116065    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-run-netns\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.116068    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.116131    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs podName:26778894-1f27-4bcc-b3de-5ae0ac3aca4c nodeName:}" failed. No retries permitted until 2026-07-10 03:32:48.616114074 +0000 UTC m=+4.130849359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs") pod "network-metrics-daemon-j6vpn" (UID: "26778894-1f27-4bcc-b3de-5ae0ac3aca4c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:48.117089 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116165    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-run-openvswitch\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116176    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bfe8cc34-69bf-4d9e-a945-b1c867b6111a-serviceca\") pod \"node-ca-xnfdx\" (UID: \"bfe8cc34-69bf-4d9e-a945-b1c867b6111a\") " pod="openshift-image-registry/node-ca-xnfdx"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116234    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-run-ovn\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116299    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-cni-netd\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116338    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d4200887-8985-4033-8fb6-5660c89ff1f7-env-overrides\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116385    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/26fcd562-4c93-4255-9a6d-0c4a362b4b72-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116400    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-log-socket\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116438    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/859d3328-63da-49c4-a871-2b45a4960764-dbus\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116450    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-systemd-units\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116519    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-var-lib-openvswitch\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116571    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/859d3328-63da-49c4-a871-2b45a4960764-kubelet-config\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116576    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d4200887-8985-4033-8fb6-5660c89ff1f7-ovnkube-config\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116661    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-node-log\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116792    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/26fcd562-4c93-4255-9a6d-0c4a362b4b72-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116451    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-slash\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.117838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116849    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfe8cc34-69bf-4d9e-a945-b1c867b6111a-host\") pod \"node-ca-xnfdx\" (UID: \"bfe8cc34-69bf-4d9e-a945-b1c867b6111a\") " pod="openshift-image-registry/node-ca-xnfdx"
Jul 10 03:32:48.118502 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116878    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/26fcd562-4c93-4255-9a6d-0c4a362b4b72-cnibin\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.118502 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.116889    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-run-systemd\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.118502 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115969    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-run-ovn-kubernetes\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.118502 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.115869    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4200887-8985-4033-8fb6-5660c89ff1f7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.118502 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.117252    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/26fcd562-4c93-4255-9a6d-0c4a362b4b72-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.118502 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.117360    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8c21afc9-23db-4b36-b482-8c244a254df6-konnectivity-ca\") pod \"konnectivity-agent-8458m\" (UID: \"8c21afc9-23db-4b36-b482-8c244a254df6\") " pod="kube-system/konnectivity-agent-8458m"
Jul 10 03:32:48.118502 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.117437    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/26fcd562-4c93-4255-9a6d-0c4a362b4b72-cni-binary-copy\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.118502 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.117448    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d4200887-8985-4033-8fb6-5660c89ff1f7-ovnkube-script-lib\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.119124 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.119099    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8c21afc9-23db-4b36-b482-8c244a254df6-agent-certs\") pod \"konnectivity-agent-8458m\" (UID: \"8c21afc9-23db-4b36-b482-8c244a254df6\") " pod="kube-system/konnectivity-agent-8458m"
Jul 10 03:32:48.119215 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.119139    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4200887-8985-4033-8fb6-5660c89ff1f7-ovn-node-metrics-cert\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.123364 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.123302    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:32:48.123364 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.123332    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:32:48.123364 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.123345    2571 projected.go:196] Error preparing data for projected volume kube-api-access-jzwqc for pod openshift-network-diagnostics/network-check-target-zkhkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:48.123612 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.123405    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc podName:ef22232c-35eb-4554-8518-ea64524eefa1 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:48.623388533 +0000 UTC m=+4.138123820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jzwqc" (UniqueName: "kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc") pod "network-check-target-zkhkn" (UID: "ef22232c-35eb-4554-8518-ea64524eefa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:48.124006 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.123939    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zwf6\" (UniqueName: \"kubernetes.io/projected/d4200887-8985-4033-8fb6-5660c89ff1f7-kube-api-access-6zwf6\") pod \"ovnkube-node-gxvf2\" (UID: \"d4200887-8985-4033-8fb6-5660c89ff1f7\") " pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.126570 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.126475    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-656sj\" (UniqueName: \"kubernetes.io/projected/26fcd562-4c93-4255-9a6d-0c4a362b4b72-kube-api-access-656sj\") pod \"multus-additional-cni-plugins-dhvb5\" (UID: \"26fcd562-4c93-4255-9a6d-0c4a362b4b72\") " pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.126943 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.126911    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z598\" (UniqueName: \"kubernetes.io/projected/bfe8cc34-69bf-4d9e-a945-b1c867b6111a-kube-api-access-9z598\") pod \"node-ca-xnfdx\" (UID: \"bfe8cc34-69bf-4d9e-a945-b1c867b6111a\") " pod="openshift-image-registry/node-ca-xnfdx"
Jul 10 03:32:48.127284 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.127261    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sghc\" (UniqueName: \"kubernetes.io/projected/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-kube-api-access-2sghc\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:48.193861 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.193823    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7dqr4"
Jul 10 03:32:48.203127 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.203098    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt"
Jul 10 03:32:48.214197 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.214169    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pmhgt"
Jul 10 03:32:48.221182 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.221158    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h6nnw"
Jul 10 03:32:48.228876 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.228857    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9p7cz"
Jul 10 03:32:48.236417 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.236391    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:32:48.244245 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.244226    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8458m"
Jul 10 03:32:48.253119 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.253088    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xnfdx"
Jul 10 03:32:48.258808 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.258786    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dhvb5"
Jul 10 03:32:48.619922 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.619816    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:48.619922 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.619894    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:48.620139 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.619999    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:48.620139 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.620017    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:48.620139 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.620071    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret podName:859d3328-63da-49c4-a871-2b45a4960764 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:49.620055174 +0000 UTC m=+5.134790456 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret") pod "global-pull-secret-syncer-vlr5v" (UID: "859d3328-63da-49c4-a871-2b45a4960764") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:48.620139 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.620087    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs podName:26778894-1f27-4bcc-b3de-5ae0ac3aca4c nodeName:}" failed. No retries permitted until 2026-07-10 03:32:49.620079287 +0000 UTC m=+5.134814569 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs") pod "network-metrics-daemon-j6vpn" (UID: "26778894-1f27-4bcc-b3de-5ae0ac3aca4c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:48.720555 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.720513    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzwqc\" (UniqueName: \"kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc\") pod \"network-check-target-zkhkn\" (UID: \"ef22232c-35eb-4554-8518-ea64524eefa1\") " pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:32:48.720759 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.720731    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:32:48.720832 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.720766    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:32:48.720832 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.720781    2571 projected.go:196] Error preparing data for projected volume kube-api-access-jzwqc for pod openshift-network-diagnostics/network-check-target-zkhkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:48.720931 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:48.720853    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc podName:ef22232c-35eb-4554-8518-ea64524eefa1 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:49.720829558 +0000 UTC m=+5.235564857 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzwqc" (UniqueName: "kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc") pod "network-check-target-zkhkn" (UID: "ef22232c-35eb-4554-8518-ea64524eefa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:48.928617 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.928554    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 03:27:47 +0000 UTC" deadline="2028-04-18 16:39:03.127819292 +0000 UTC"
Jul 10 03:32:48.928617 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:48.928613    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15565h6m14.199209537s"
Jul 10 03:32:49.001038 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:49.000946    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod638ec913_d0a6_4937_bc25_adc4d6fc5897.slice/crio-83d7080b9bb116cad0f63ab2061d7bd591f21870e118d36f0cd4b5a013727ebc WatchSource:0}: Error finding container 83d7080b9bb116cad0f63ab2061d7bd591f21870e118d36f0cd4b5a013727ebc: Status 404 returned error can't find the container with id 83d7080b9bb116cad0f63ab2061d7bd591f21870e118d36f0cd4b5a013727ebc
Jul 10 03:32:49.001977 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:49.001948    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebc252f0_1441_4bf2_a593_8eabd8aba528.slice/crio-1f97d5f92fc28a309bb32a835cbd620a54fe6802f73c57c4d8bc069e0b6ab051 WatchSource:0}: Error finding container 1f97d5f92fc28a309bb32a835cbd620a54fe6802f73c57c4d8bc069e0b6ab051: Status 404 returned error can't find the container with id 1f97d5f92fc28a309bb32a835cbd620a54fe6802f73c57c4d8bc069e0b6ab051
Jul 10 03:32:49.002963 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:49.002760    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaa2bc4a_1a56_4648_a3ee_703e4258204b.slice/crio-015e811d8d05c2aa6fe4243592bdddd806883c5f126995e97eca62735d16d732 WatchSource:0}: Error finding container 015e811d8d05c2aa6fe4243592bdddd806883c5f126995e97eca62735d16d732: Status 404 returned error can't find the container with id 015e811d8d05c2aa6fe4243592bdddd806883c5f126995e97eca62735d16d732
Jul 10 03:32:49.006458 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:49.006435    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc014e314_0ed8_4ecc_8c60_5447472f4121.slice/crio-8d36a312c0bb452fed37f2aad80c105d5fa6d0dd6fdaefd8044bb6647a4b14be WatchSource:0}: Error finding container 8d36a312c0bb452fed37f2aad80c105d5fa6d0dd6fdaefd8044bb6647a4b14be: Status 404 returned error can't find the container with id 8d36a312c0bb452fed37f2aad80c105d5fa6d0dd6fdaefd8044bb6647a4b14be
Jul 10 03:32:49.007321 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:49.007297    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c21afc9_23db_4b36_b482_8c244a254df6.slice/crio-c2c795a1cc2ae98fbb7c1a693ffbd0979315f1c3f4a7bbb9526f8debc34d3d85 WatchSource:0}: Error finding container c2c795a1cc2ae98fbb7c1a693ffbd0979315f1c3f4a7bbb9526f8debc34d3d85: Status 404 returned error can't find the container with id c2c795a1cc2ae98fbb7c1a693ffbd0979315f1c3f4a7bbb9526f8debc34d3d85
Jul 10 03:32:49.008713 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:49.008687    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b1d7593_1d0c_4166_8dc8_733dc6c4f55c.slice/crio-9560dc564e8d9b479edd09fd71770e1e09737ca1d68509043658ab8d9c4aebcf WatchSource:0}: Error finding container 9560dc564e8d9b479edd09fd71770e1e09737ca1d68509043658ab8d9c4aebcf: Status 404 returned error can't find the container with id 9560dc564e8d9b479edd09fd71770e1e09737ca1d68509043658ab8d9c4aebcf
Jul 10 03:32:49.010230 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:49.010200    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26fcd562_4c93_4255_9a6d_0c4a362b4b72.slice/crio-9f252e46a0e0469f68629601ec8f4db67e650f2fa81020551ead129eef11ef15 WatchSource:0}: Error finding container 9f252e46a0e0469f68629601ec8f4db67e650f2fa81020551ead129eef11ef15: Status 404 returned error can't find the container with id 9f252e46a0e0469f68629601ec8f4db67e650f2fa81020551ead129eef11ef15
Jul 10 03:32:49.011653 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:49.011451    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe8cc34_69bf_4d9e_a945_b1c867b6111a.slice/crio-c7e9b4033e8c536ee2d4353fcd088ebb57462e8b412fadedc39b451725dd01d1 WatchSource:0}: Error finding container c7e9b4033e8c536ee2d4353fcd088ebb57462e8b412fadedc39b451725dd01d1: Status 404 returned error can't find the container with id c7e9b4033e8c536ee2d4353fcd088ebb57462e8b412fadedc39b451725dd01d1
Jul 10 03:32:49.012902 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:32:49.012877    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4200887_8985_4033_8fb6_5660c89ff1f7.slice/crio-a0d1a4bfa870befba7ebb937663c491f15b53dac61f7ed3dfce18c9a1ee4cc63 WatchSource:0}: Error finding container a0d1a4bfa870befba7ebb937663c491f15b53dac61f7ed3dfce18c9a1ee4cc63: Status 404 returned error can't find the container with id a0d1a4bfa870befba7ebb937663c491f15b53dac61f7ed3dfce18c9a1ee4cc63
Jul 10 03:32:49.121129 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.120934    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:32:49.121324 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:49.121242    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:32:49.133866 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.133824    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8458m" event={"ID":"8c21afc9-23db-4b36-b482-8c244a254df6","Type":"ContainerStarted","Data":"c2c795a1cc2ae98fbb7c1a693ffbd0979315f1c3f4a7bbb9526f8debc34d3d85"}
Jul 10 03:32:49.136733 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.136697    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2" event={"ID":"d4200887-8985-4033-8fb6-5660c89ff1f7","Type":"ContainerStarted","Data":"a0d1a4bfa870befba7ebb937663c491f15b53dac61f7ed3dfce18c9a1ee4cc63"}
Jul 10 03:32:49.141467 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.141397    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h6nnw" event={"ID":"c014e314-0ed8-4ecc-8c60-5447472f4121","Type":"ContainerStarted","Data":"8d36a312c0bb452fed37f2aad80c105d5fa6d0dd6fdaefd8044bb6647a4b14be"}
Jul 10 03:32:49.143337 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.143274    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pmhgt" event={"ID":"ebc252f0-1441-4bf2-a593-8eabd8aba528","Type":"ContainerStarted","Data":"1f97d5f92fc28a309bb32a835cbd620a54fe6802f73c57c4d8bc069e0b6ab051"}
Jul 10 03:32:49.144439 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.144413    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7dqr4" event={"ID":"638ec913-d0a6-4937-bc25-adc4d6fc5897","Type":"ContainerStarted","Data":"83d7080b9bb116cad0f63ab2061d7bd591f21870e118d36f0cd4b5a013727ebc"}
Jul 10 03:32:49.147418 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.147391    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal" event={"ID":"0b3b8930aa1846297e408b672c61a6ab","Type":"ContainerStarted","Data":"02bf1a40b1f6f272ba438f698ba471b086aec8046255616b5b0a7cbaffb8cc9a"}
Jul 10 03:32:49.151096 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.150993    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dhvb5" event={"ID":"26fcd562-4c93-4255-9a6d-0c4a362b4b72","Type":"ContainerStarted","Data":"9f252e46a0e0469f68629601ec8f4db67e650f2fa81020551ead129eef11ef15"}
Jul 10 03:32:49.152698 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.152672    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9p7cz" event={"ID":"4b1d7593-1d0c-4166-8dc8-733dc6c4f55c","Type":"ContainerStarted","Data":"9560dc564e8d9b479edd09fd71770e1e09737ca1d68509043658ab8d9c4aebcf"}
Jul 10 03:32:49.155064 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.155040    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt" event={"ID":"baa2bc4a-1a56-4648-a3ee-703e4258204b","Type":"ContainerStarted","Data":"015e811d8d05c2aa6fe4243592bdddd806883c5f126995e97eca62735d16d732"}
Jul 10 03:32:49.156809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.156782    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xnfdx" event={"ID":"bfe8cc34-69bf-4d9e-a945-b1c867b6111a","Type":"ContainerStarted","Data":"c7e9b4033e8c536ee2d4353fcd088ebb57462e8b412fadedc39b451725dd01d1"}
Jul 10 03:32:49.162014 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.161954    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-142.ec2.internal" podStartSLOduration=2.161939791 podStartE2EDuration="2.161939791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:49.16179046 +0000 UTC m=+4.676525768" watchObservedRunningTime="2026-07-10 03:32:49.161939791 +0000 UTC m=+4.676675098"
Jul 10 03:32:49.628957 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.628867    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:49.628957 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.628955    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:49.629203 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:49.629089    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:49.629203 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:49.629154    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs podName:26778894-1f27-4bcc-b3de-5ae0ac3aca4c nodeName:}" failed. No retries permitted until 2026-07-10 03:32:51.629135143 +0000 UTC m=+7.143870430 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs") pod "network-metrics-daemon-j6vpn" (UID: "26778894-1f27-4bcc-b3de-5ae0ac3aca4c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:49.629622 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:49.629600    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:49.629699 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:49.629665    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret podName:859d3328-63da-49c4-a871-2b45a4960764 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:51.629647664 +0000 UTC m=+7.144382952 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret") pod "global-pull-secret-syncer-vlr5v" (UID: "859d3328-63da-49c4-a871-2b45a4960764") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:49.730238 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:49.730194    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzwqc\" (UniqueName: \"kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc\") pod \"network-check-target-zkhkn\" (UID: \"ef22232c-35eb-4554-8518-ea64524eefa1\") " pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:32:49.730448 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:49.730422    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:32:49.730448 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:49.730443    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:32:49.730601 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:49.730458    2571 projected.go:196] Error preparing data for projected volume kube-api-access-jzwqc for pod openshift-network-diagnostics/network-check-target-zkhkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:49.730601 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:49.730542    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc podName:ef22232c-35eb-4554-8518-ea64524eefa1 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:51.730520793 +0000 UTC m=+7.245256090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzwqc" (UniqueName: "kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc") pod "network-check-target-zkhkn" (UID: "ef22232c-35eb-4554-8518-ea64524eefa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:50.120242 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:50.120145    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:50.120763 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:50.120314    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:32:50.120763 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:50.120338    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:50.120763 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:50.120454    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:32:50.194997 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:50.194862    2571 generic.go:356] "Generic (PLEG): container finished" podID="343c2c99cf08d1b78e530233805856b4" containerID="3dfc5bb5e9ec46f44cb9a68167356bfd752793b0b72aea4b7d8ffb6bb19baa97" exitCode=0
Jul 10 03:32:50.195830 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:50.195453    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" event={"ID":"343c2c99cf08d1b78e530233805856b4","Type":"ContainerDied","Data":"3dfc5bb5e9ec46f44cb9a68167356bfd752793b0b72aea4b7d8ffb6bb19baa97"}
Jul 10 03:32:51.122737 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:51.122698    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:32:51.123237 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:51.122839    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:32:51.223720 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:51.222946    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" event={"ID":"343c2c99cf08d1b78e530233805856b4","Type":"ContainerStarted","Data":"62762ad5e826cf3412b3e21704899cfe8c871191432feccef434103d4394a86b"}
Jul 10 03:32:51.645871 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:51.645825    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:51.646054 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:51.645913    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:51.646114 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:51.646081    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:51.646185 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:51.646151    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret podName:859d3328-63da-49c4-a871-2b45a4960764 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:55.646130887 +0000 UTC m=+11.160866176 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret") pod "global-pull-secret-syncer-vlr5v" (UID: "859d3328-63da-49c4-a871-2b45a4960764") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:51.646260 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:51.646223    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:51.646260 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:51.646256    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs podName:26778894-1f27-4bcc-b3de-5ae0ac3aca4c nodeName:}" failed. No retries permitted until 2026-07-10 03:32:55.646244924 +0000 UTC m=+11.160980215 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs") pod "network-metrics-daemon-j6vpn" (UID: "26778894-1f27-4bcc-b3de-5ae0ac3aca4c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:51.748195 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:51.747265    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzwqc\" (UniqueName: \"kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc\") pod \"network-check-target-zkhkn\" (UID: \"ef22232c-35eb-4554-8518-ea64524eefa1\") " pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:32:51.748195 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:51.747674    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:32:51.748195 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:51.747699    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:32:51.748195 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:51.747714    2571 projected.go:196] Error preparing data for projected volume kube-api-access-jzwqc for pod openshift-network-diagnostics/network-check-target-zkhkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:51.748195 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:51.747777    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc podName:ef22232c-35eb-4554-8518-ea64524eefa1 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:55.74775694 +0000 UTC m=+11.262492226 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzwqc" (UniqueName: "kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc") pod "network-check-target-zkhkn" (UID: "ef22232c-35eb-4554-8518-ea64524eefa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:52.120616 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:52.119847    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:52.120616 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:52.120004    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:32:52.120616 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:52.120402    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:52.120616 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:52.120500    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:32:53.120457 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:53.120421    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:32:53.120909 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:53.120594    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:32:54.120955 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:54.120369    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:54.120955 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:54.120376    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:54.120955 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:54.120542    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:32:54.120955 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:54.120651    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:32:55.121261 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:55.121229    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:32:55.121783 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:55.121325    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:32:55.678841 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:55.678515    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:55.678841 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:55.678606    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:55.678841 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:55.678752    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:55.678841 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:55.678759    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:55.678841 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:55.678818    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs podName:26778894-1f27-4bcc-b3de-5ae0ac3aca4c nodeName:}" failed. No retries permitted until 2026-07-10 03:33:03.678800478 +0000 UTC m=+19.193535764 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs") pod "network-metrics-daemon-j6vpn" (UID: "26778894-1f27-4bcc-b3de-5ae0ac3aca4c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:32:55.678841 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:55.678840    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret podName:859d3328-63da-49c4-a871-2b45a4960764 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:03.678829617 +0000 UTC m=+19.193564904 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret") pod "global-pull-secret-syncer-vlr5v" (UID: "859d3328-63da-49c4-a871-2b45a4960764") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:32:55.779852 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:55.779805    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzwqc\" (UniqueName: \"kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc\") pod \"network-check-target-zkhkn\" (UID: \"ef22232c-35eb-4554-8518-ea64524eefa1\") " pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:32:55.780045 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:55.780018    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:32:55.780045 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:55.780050    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:32:55.780234 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:55.780064    2571 projected.go:196] Error preparing data for projected volume kube-api-access-jzwqc for pod openshift-network-diagnostics/network-check-target-zkhkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:55.780234 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:55.780132    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc podName:ef22232c-35eb-4554-8518-ea64524eefa1 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:03.780112052 +0000 UTC m=+19.294847338 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzwqc" (UniqueName: "kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc") pod "network-check-target-zkhkn" (UID: "ef22232c-35eb-4554-8518-ea64524eefa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:32:56.120699 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:56.120620    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:56.120699 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:56.120663    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:56.120919 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:56.120758    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:32:56.120919 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:56.120879    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:32:57.120893 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:57.120857    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:32:57.121324 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:57.120964    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:32:58.120400 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:58.120200    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:32:58.120400 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:58.120230    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:32:58.120400 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:58.120348    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:32:58.120703 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:58.120476    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:32:59.120833 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:32:59.120348    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:32:59.120833 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:32:59.120499    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:33:00.120520 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.120480    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:00.120700 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.120530    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:00.120700 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:00.120634    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:33:00.120825 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:00.120760    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:33:00.249900 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.249861    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt" event={"ID":"baa2bc4a-1a56-4648-a3ee-703e4258204b","Type":"ContainerStarted","Data":"8084fb1ce1527a48054d7ad928ba1fb604370634c5e516efd93295371af7f5b2"}
Jul 10 03:33:00.251662 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.251630    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xnfdx" event={"ID":"bfe8cc34-69bf-4d9e-a945-b1c867b6111a","Type":"ContainerStarted","Data":"b05f2d243ba72da4a9aef9befa25027940922d89e5a4977a5418fcf806e92e79"}
Jul 10 03:33:00.253369 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.253340    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8458m" event={"ID":"8c21afc9-23db-4b36-b482-8c244a254df6","Type":"ContainerStarted","Data":"d27687a9f46caa7f5031e7eb8fd11ed88c5d24017e4055e024be0816a74c229f"}
Jul 10 03:33:00.255190 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.255161    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h6nnw" event={"ID":"c014e314-0ed8-4ecc-8c60-5447472f4121","Type":"ContainerStarted","Data":"3e5e1901fe450eb72bbaecad22043d4722ac072a4b97a0db212da931fa98712d"}
Jul 10 03:33:00.256997 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.256967    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dhvb5" event={"ID":"26fcd562-4c93-4255-9a6d-0c4a362b4b72","Type":"ContainerStarted","Data":"759cdd7700075f141fda765c59f13ac1eb55b553aaf4531f01443d245dc85ea1"}
Jul 10 03:33:00.258445 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.258397    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9p7cz" event={"ID":"4b1d7593-1d0c-4166-8dc8-733dc6c4f55c","Type":"ContainerStarted","Data":"0e053d38a2bd4fdc9de4eb3a2fdade932ee05fab205688ec21b976ae7d0f985b"}
Jul 10 03:33:00.272091 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.272041    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xnfdx" podStartSLOduration=4.852153327 podStartE2EDuration="15.272026226s" totalImagesPullingTime="10.419872899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:00.271676074 +0000 UTC m=+15.786411377" watchObservedRunningTime="2026-07-10 03:33:00.272026226 +0000 UTC m=+15.786761635"
Jul 10 03:33:00.272783 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.272737    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-142.ec2.internal" podStartSLOduration=13.272721427 podStartE2EDuration="13.272721427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:51.239345763 +0000 UTC m=+6.754081071" watchObservedRunningTime="2026-07-10 03:33:00.272721427 +0000 UTC m=+15.787456732"
Jul 10 03:33:00.288649 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.288586    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h6nnw" podStartSLOduration=4.8311337210000005 podStartE2EDuration="15.288566488s" totalImagesPullingTime="10.457432767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:00.287713707 +0000 UTC m=+15.802449011" watchObservedRunningTime="2026-07-10 03:33:00.288566488 +0000 UTC m=+15.803301791"
Jul 10 03:33:00.304500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.304423    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9p7cz" podStartSLOduration=4.881028055 podStartE2EDuration="15.304406253s" totalImagesPullingTime="10.423378198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:00.303832532 +0000 UTC m=+15.818567836" watchObservedRunningTime="2026-07-10 03:33:00.304406253 +0000 UTC m=+15.819141555"
Jul 10 03:33:00.315570 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.315542    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8458m"
Jul 10 03:33:00.316090 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.316064    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8458m"
Jul 10 03:33:00.320628 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:00.320590    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8458m" podStartSLOduration=4.928742083 podStartE2EDuration="15.32057795s" totalImagesPullingTime="10.391835867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:00.3202263 +0000 UTC m=+15.834961603" watchObservedRunningTime="2026-07-10 03:33:00.32057795 +0000 UTC m=+15.835313253"
Jul 10 03:33:01.120266 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:01.120228    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:01.120414 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:01.120385    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:33:01.262121 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:01.262066    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pmhgt" event={"ID":"ebc252f0-1441-4bf2-a593-8eabd8aba528","Type":"ContainerStarted","Data":"aaa5aa4f56ef21ecd7e8c3e860521047d3f49162a0bf6decfbc5d9eb5a042e2d"}
Jul 10 03:33:01.263519 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:01.263493    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8458m"
Jul 10 03:33:01.263623 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:01.263550    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8458m"
Jul 10 03:33:01.279611 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:01.279512    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pmhgt" podStartSLOduration=5.875911163 podStartE2EDuration="16.27949093s" totalImagesPullingTime="10.403579767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:01.279095027 +0000 UTC m=+16.793830330" watchObservedRunningTime="2026-07-10 03:33:01.27949093 +0000 UTC m=+16.794226236"
Jul 10 03:33:02.120285 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:02.120241    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:02.120471 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:02.120370    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:33:02.120471 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:02.120419    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:02.120577 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:02.120553    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:33:03.120862 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:03.120824    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:03.121333 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:03.120963    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:33:03.741175 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:03.741136    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:03.741364 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:03.741241    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:03.741364 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:03.741276    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:33:03.741364 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:03.741357    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs podName:26778894-1f27-4bcc-b3de-5ae0ac3aca4c nodeName:}" failed. No retries permitted until 2026-07-10 03:33:19.741338381 +0000 UTC m=+35.256073665 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs") pod "network-metrics-daemon-j6vpn" (UID: "26778894-1f27-4bcc-b3de-5ae0ac3aca4c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:33:03.741583 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:03.741368    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:33:03.741583 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:03.741418    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret podName:859d3328-63da-49c4-a871-2b45a4960764 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:19.741403328 +0000 UTC m=+35.256138611 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret") pod "global-pull-secret-syncer-vlr5v" (UID: "859d3328-63da-49c4-a871-2b45a4960764") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:33:03.841989 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:03.841947    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzwqc\" (UniqueName: \"kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc\") pod \"network-check-target-zkhkn\" (UID: \"ef22232c-35eb-4554-8518-ea64524eefa1\") " pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:03.842156 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:03.842121    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:33:03.842156 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:03.842145    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:33:03.842156 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:03.842157    2571 projected.go:196] Error preparing data for projected volume kube-api-access-jzwqc for pod openshift-network-diagnostics/network-check-target-zkhkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:33:03.842317 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:03.842231    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc podName:ef22232c-35eb-4554-8518-ea64524eefa1 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:19.842205428 +0000 UTC m=+35.356940713 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzwqc" (UniqueName: "kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc") pod "network-check-target-zkhkn" (UID: "ef22232c-35eb-4554-8518-ea64524eefa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:33:04.120744 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:04.120657    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:04.120906 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:04.120774    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:33:04.120906 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:04.120833    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:04.121435 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:04.120934    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:33:04.270015 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:04.269978    2571 generic.go:356] "Generic (PLEG): container finished" podID="26fcd562-4c93-4255-9a6d-0c4a362b4b72" containerID="759cdd7700075f141fda765c59f13ac1eb55b553aaf4531f01443d245dc85ea1" exitCode=0
Jul 10 03:33:04.270180 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:04.270038    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dhvb5" event={"ID":"26fcd562-4c93-4255-9a6d-0c4a362b4b72","Type":"ContainerDied","Data":"759cdd7700075f141fda765c59f13ac1eb55b553aaf4531f01443d245dc85ea1"}
Jul 10 03:33:05.120924 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:05.120885    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:05.121416 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:05.120988    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:33:06.120179 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:06.120143    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:06.120348 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:06.120200    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:06.120348 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:06.120296    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:33:06.120474 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:06.120431    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:33:07.120360 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:07.120324    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:07.120817 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:07.120455    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:33:08.119998 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:08.119826    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:08.120223 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:08.119839    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:08.120223 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:08.120098    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:33:08.120223 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:08.120167    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:33:08.279864 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:08.279822    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2" event={"ID":"d4200887-8985-4033-8fb6-5660c89ff1f7","Type":"ContainerStarted","Data":"dd7af35c4d3d825b2e8ecd68dcbd81f06c875c22ef32607224c188bc660281f5"}
Jul 10 03:33:08.281575 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:08.281544    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7dqr4" event={"ID":"638ec913-d0a6-4937-bc25-adc4d6fc5897","Type":"ContainerStarted","Data":"845081a6bcea1c766980433faa4ceec64691681a68ece75645efc68716e6f972"}
Jul 10 03:33:08.301715 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:08.301648    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7dqr4" podStartSLOduration=4.180212121 podStartE2EDuration="23.301626682s" totalImagesPullingTime="19.121414561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:08.300409171 +0000 UTC m=+23.815144475" watchObservedRunningTime="2026-07-10 03:33:08.301626682 +0000 UTC m=+23.816361985"
Jul 10 03:33:08.549891 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:08.549749    2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 03:33:08.980579 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:08.980450    2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T03:33:08.549887415Z","UUID":"0d67ca76-f278-4bb3-82b5-371397b95415","Handler":null,"Name":"","Endpoint":""}
Jul 10 03:33:08.982301 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:08.982282    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 03:33:08.982414 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:08.982324    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 03:33:09.120486 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:09.120396    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:09.120643 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:09.120552    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:33:09.289608 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:09.289566    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2" event={"ID":"d4200887-8985-4033-8fb6-5660c89ff1f7","Type":"ContainerStarted","Data":"3bb4adf75180c05bd40c489c1db58186ad3de287b3a6e69dcfc34f749b53689c"}
Jul 10 03:33:09.290176 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:09.289697    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2" event={"ID":"d4200887-8985-4033-8fb6-5660c89ff1f7","Type":"ContainerStarted","Data":"064c90960f38a5d0e0bfc252ed85df8f6cf584e2c8bc7e1b1b35f26f57e30a49"}
Jul 10 03:33:09.290176 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:09.289750    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2" event={"ID":"d4200887-8985-4033-8fb6-5660c89ff1f7","Type":"ContainerStarted","Data":"dab6cc68db663cf982f30c7ce3767d63ccd12fced945f8499c83776913da1281"}
Jul 10 03:33:09.290176 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:09.289797    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2" event={"ID":"d4200887-8985-4033-8fb6-5660c89ff1f7","Type":"ContainerStarted","Data":"fffd93bef2c36fe876aa877b37bed455ec6647d118bfc7604ffce678c17e5999"}
Jul 10 03:33:09.290176 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:09.289842    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2" event={"ID":"d4200887-8985-4033-8fb6-5660c89ff1f7","Type":"ContainerStarted","Data":"9920203a864421535dd86f9f1cb5a0446c0237134f0f94e2a3596ba94a19feda"}
Jul 10 03:33:09.291540 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:09.291514    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt" event={"ID":"baa2bc4a-1a56-4648-a3ee-703e4258204b","Type":"ContainerStarted","Data":"fea6abb71c8f375e738af1bb8676a345c02e596853f5b19ab9de0c06288ddac2"}
Jul 10 03:33:10.120234 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:10.120147    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:10.120388 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:10.120258    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:33:10.120388 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:10.120317    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:10.120531 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:10.120443    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:33:10.295487 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:10.295440    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt" event={"ID":"baa2bc4a-1a56-4648-a3ee-703e4258204b","Type":"ContainerStarted","Data":"4cbd6d83a50b5acd3593bf5d06680181a241b7f8291d7855ebe010dfec04bb45"}
Jul 10 03:33:10.313739 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:10.313693    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zdntt" podStartSLOduration=4.897624463 podStartE2EDuration="25.313678395s" totalImagesPullingTime="20.416053932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:10.313163989 +0000 UTC m=+25.827899292" watchObservedRunningTime="2026-07-10 03:33:10.313678395 +0000 UTC m=+25.828413727"
Jul 10 03:33:11.120418 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:11.120382    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:11.120623 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:11.120528    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:33:11.300682 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:11.300646    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2" event={"ID":"d4200887-8985-4033-8fb6-5660c89ff1f7","Type":"ContainerStarted","Data":"cb12805a5c46f10bde99b70792b23137740f1665124a89213728b8cd8e81d292"}
Jul 10 03:33:12.119958 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:12.119919    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:12.120150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:12.119919    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:12.120150 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:12.120044    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:33:12.120150 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:12.120123    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:33:13.123631 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:13.123573    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:13.124079 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:13.123680    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:33:13.307901 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:13.307713    2571 generic.go:356] "Generic (PLEG): container finished" podID="26fcd562-4c93-4255-9a6d-0c4a362b4b72" containerID="8c7a672f983533957ca2e8545394ebf91b6cec44dd7e71d96091e1f96be7051e" exitCode=0
Jul 10 03:33:13.308055 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:13.307788    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dhvb5" event={"ID":"26fcd562-4c93-4255-9a6d-0c4a362b4b72","Type":"ContainerDied","Data":"8c7a672f983533957ca2e8545394ebf91b6cec44dd7e71d96091e1f96be7051e"}
Jul 10 03:33:13.311511 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:13.311485    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2" event={"ID":"d4200887-8985-4033-8fb6-5660c89ff1f7","Type":"ContainerStarted","Data":"5265b05dd82474dc66a319c63e735cb7c55d64ce085151a7ab185ce6723e87b6"}
Jul 10 03:33:13.311813 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:13.311792    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:33:13.327623 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:13.327593    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:33:13.351497 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:13.351430    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2" podStartSLOduration=9.241109844 podStartE2EDuration="28.351413047s" totalImagesPullingTime="19.110303203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:13.350340782 +0000 UTC m=+28.865076167" watchObservedRunningTime="2026-07-10 03:33:13.351413047 +0000 UTC m=+28.866148349"
Jul 10 03:33:14.119778 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:14.119756    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:14.119913 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:14.119889    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:33:14.119985 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:14.119941    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:14.120087 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:14.120063    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:33:14.318144 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:14.318105    2571 generic.go:356] "Generic (PLEG): container finished" podID="26fcd562-4c93-4255-9a6d-0c4a362b4b72" containerID="d9726e1c7fc8c977680a9e4b656eabd05961ef5062f8e01d80fdcc4001cb6ae5" exitCode=0
Jul 10 03:33:14.320174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:14.319325    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dhvb5" event={"ID":"26fcd562-4c93-4255-9a6d-0c4a362b4b72","Type":"ContainerDied","Data":"d9726e1c7fc8c977680a9e4b656eabd05961ef5062f8e01d80fdcc4001cb6ae5"}
Jul 10 03:33:14.320174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:14.319382    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:33:14.320174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:14.319960    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:33:14.349034 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:14.349000    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:33:14.984150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:14.984117    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j6vpn"]
Jul 10 03:33:14.984319 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:14.984304    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:14.984427 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:14.984407    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:33:14.987190 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:14.987010    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zkhkn"]
Jul 10 03:33:14.987190 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:14.987139    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:14.987374 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:14.987243    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:33:14.987737 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:14.987711    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vlr5v"]
Jul 10 03:33:14.987838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:14.987807    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:14.988016 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:14.987906    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:33:15.323277 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:15.323064    2571 generic.go:356] "Generic (PLEG): container finished" podID="26fcd562-4c93-4255-9a6d-0c4a362b4b72" containerID="0243ae1e4ac138e3fadaa45aa2e92002416133f55a1c5d1430f1188ec1d0b7f4" exitCode=0
Jul 10 03:33:15.323625 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:15.323126    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dhvb5" event={"ID":"26fcd562-4c93-4255-9a6d-0c4a362b4b72","Type":"ContainerDied","Data":"0243ae1e4ac138e3fadaa45aa2e92002416133f55a1c5d1430f1188ec1d0b7f4"}
Jul 10 03:33:16.119927 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:16.119894    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:16.120080 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:16.120001    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:33:17.120786 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:17.120755    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:17.121320 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:17.120754    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:17.121320 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:17.120908    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:33:17.121320 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:17.120943    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:33:18.120200 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:18.120165    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:18.120383 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:18.120271    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:33:19.120598 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:19.120509    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:19.120598 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:19.120527    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:19.121285 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:19.120650    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlr5v" podUID="859d3328-63da-49c4-a871-2b45a4960764"
Jul 10 03:33:19.121285 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:19.120793    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:33:19.763263 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:19.763221    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:19.763443 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:19.763288    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:19.763443 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:19.763367    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:33:19.763563 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:19.763447    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs podName:26778894-1f27-4bcc-b3de-5ae0ac3aca4c nodeName:}" failed. No retries permitted until 2026-07-10 03:33:51.763425325 +0000 UTC m=+67.278160618 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs") pod "network-metrics-daemon-j6vpn" (UID: "26778894-1f27-4bcc-b3de-5ae0ac3aca4c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 03:33:19.763563 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:19.763378    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 03:33:19.763563 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:19.763537    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret podName:859d3328-63da-49c4-a871-2b45a4960764 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:51.763516356 +0000 UTC m=+67.278251656 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret") pod "global-pull-secret-syncer-vlr5v" (UID: "859d3328-63da-49c4-a871-2b45a4960764") : object "kube-system"/"original-pull-secret" not registered
Jul 10 03:33:19.864585 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:19.864546    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzwqc\" (UniqueName: \"kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc\") pod \"network-check-target-zkhkn\" (UID: \"ef22232c-35eb-4554-8518-ea64524eefa1\") " pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:19.864770 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:19.864719    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 03:33:19.864770 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:19.864747    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 03:33:19.864770 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:19.864761    2571 projected.go:196] Error preparing data for projected volume kube-api-access-jzwqc for pod openshift-network-diagnostics/network-check-target-zkhkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:33:19.864963 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:19.864823    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc podName:ef22232c-35eb-4554-8518-ea64524eefa1 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:51.864799726 +0000 UTC m=+67.379535029 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzwqc" (UniqueName: "kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc") pod "network-check-target-zkhkn" (UID: "ef22232c-35eb-4554-8518-ea64524eefa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 03:33:20.120102 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.120023    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:20.120254 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:20.120146    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkhkn" podUID="ef22232c-35eb-4554-8518-ea64524eefa1"
Jul 10 03:33:20.622892 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.622868    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-142.ec2.internal" event="NodeReady"
Jul 10 03:33:20.623302 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.622981    2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 03:33:20.666714 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.666684    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wszvx"]
Jul 10 03:33:20.682852 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.682813    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hdnl8"]
Jul 10 03:33:20.683024 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.682947    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:20.685835 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.685803    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dk2pk\""
Jul 10 03:33:20.686155 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.686139    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 03:33:20.686226 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.686208    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 03:33:20.702513 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.702491    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wszvx"]
Jul 10 03:33:20.702638 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.702531    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hdnl8"]
Jul 10 03:33:20.702638 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.702541    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hdnl8"
Jul 10 03:33:20.704853 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.704822    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7txf6\""
Jul 10 03:33:20.704964 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.704855    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 03:33:20.704964 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.704863    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 03:33:20.704964 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.704855    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 03:33:20.770910 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.770883    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3a9c521-10bc-4303-beee-302cee1be142-config-volume\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:20.771051 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.770913    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59w8s\" (UniqueName: \"kubernetes.io/projected/a3a9c521-10bc-4303-beee-302cee1be142-kube-api-access-59w8s\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:20.771051 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.771016    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3a9c521-10bc-4303-beee-302cee1be142-tmp-dir\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:20.771154 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.771066    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:20.871660 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.871620    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:20.871843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.871684    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3a9c521-10bc-4303-beee-302cee1be142-config-volume\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:20.871843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.871704    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59w8s\" (UniqueName: \"kubernetes.io/projected/a3a9c521-10bc-4303-beee-302cee1be142-kube-api-access-59w8s\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:20.871843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.871736    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwrmd\" (UniqueName: \"kubernetes.io/projected/86afb721-e30b-48ed-b932-7ceca0182679-kube-api-access-hwrmd\") pod \"ingress-canary-hdnl8\" (UID: \"86afb721-e30b-48ed-b932-7ceca0182679\") " pod="openshift-ingress-canary/ingress-canary-hdnl8"
Jul 10 03:33:20.871843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.871766    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86afb721-e30b-48ed-b932-7ceca0182679-cert\") pod \"ingress-canary-hdnl8\" (UID: \"86afb721-e30b-48ed-b932-7ceca0182679\") " pod="openshift-ingress-canary/ingress-canary-hdnl8"
Jul 10 03:33:20.871843 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:20.871783    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:33:20.872093 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:20.871855    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls podName:a3a9c521-10bc-4303-beee-302cee1be142 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:21.371833797 +0000 UTC m=+36.886569084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls") pod "dns-default-wszvx" (UID: "a3a9c521-10bc-4303-beee-302cee1be142") : secret "dns-default-metrics-tls" not found
Jul 10 03:33:20.872093 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.871869    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3a9c521-10bc-4303-beee-302cee1be142-tmp-dir\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:20.872205 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.872176    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3a9c521-10bc-4303-beee-302cee1be142-tmp-dir\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:20.872362 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.872340    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3a9c521-10bc-4303-beee-302cee1be142-config-volume\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:20.881968 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.881933    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59w8s\" (UniqueName: \"kubernetes.io/projected/a3a9c521-10bc-4303-beee-302cee1be142-kube-api-access-59w8s\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:20.973068 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.973033    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwrmd\" (UniqueName: \"kubernetes.io/projected/86afb721-e30b-48ed-b932-7ceca0182679-kube-api-access-hwrmd\") pod \"ingress-canary-hdnl8\" (UID: \"86afb721-e30b-48ed-b932-7ceca0182679\") " pod="openshift-ingress-canary/ingress-canary-hdnl8"
Jul 10 03:33:20.973234 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.973088    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86afb721-e30b-48ed-b932-7ceca0182679-cert\") pod \"ingress-canary-hdnl8\" (UID: \"86afb721-e30b-48ed-b932-7ceca0182679\") " pod="openshift-ingress-canary/ingress-canary-hdnl8"
Jul 10 03:33:20.975602 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.975572    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86afb721-e30b-48ed-b932-7ceca0182679-cert\") pod \"ingress-canary-hdnl8\" (UID: \"86afb721-e30b-48ed-b932-7ceca0182679\") " pod="openshift-ingress-canary/ingress-canary-hdnl8"
Jul 10 03:33:20.980670 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:20.980645    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwrmd\" (UniqueName: \"kubernetes.io/projected/86afb721-e30b-48ed-b932-7ceca0182679-kube-api-access-hwrmd\") pod \"ingress-canary-hdnl8\" (UID: \"86afb721-e30b-48ed-b932-7ceca0182679\") " pod="openshift-ingress-canary/ingress-canary-hdnl8"
Jul 10 03:33:21.010848 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:21.010809    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hdnl8"
Jul 10 03:33:21.120210 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:21.120034    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:21.120814 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:21.120034    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:21.123146 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:21.123114    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sg5lf\""
Jul 10 03:33:21.123813 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:21.123447    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 03:33:21.123813 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:21.123672    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 03:33:21.189128 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:21.189098    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hdnl8"]
Jul 10 03:33:21.192576 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:33:21.192552    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86afb721_e30b_48ed_b932_7ceca0182679.slice/crio-0fa991199eaf1a8ed654e79e5c00acfdacdce23b6844d67cc4829c62bbb2fa01 WatchSource:0}: Error finding container 0fa991199eaf1a8ed654e79e5c00acfdacdce23b6844d67cc4829c62bbb2fa01: Status 404 returned error can't find the container with id 0fa991199eaf1a8ed654e79e5c00acfdacdce23b6844d67cc4829c62bbb2fa01
Jul 10 03:33:21.339192 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:21.339157    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hdnl8" event={"ID":"86afb721-e30b-48ed-b932-7ceca0182679","Type":"ContainerStarted","Data":"0fa991199eaf1a8ed654e79e5c00acfdacdce23b6844d67cc4829c62bbb2fa01"}
Jul 10 03:33:21.341615 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:21.341588    2571 generic.go:356] "Generic (PLEG): container finished" podID="26fcd562-4c93-4255-9a6d-0c4a362b4b72" containerID="45cca818e240a0080084d27363499bd78019af3256f6cb3467f881433e3be1f6" exitCode=0
Jul 10 03:33:21.341749 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:21.341625    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dhvb5" event={"ID":"26fcd562-4c93-4255-9a6d-0c4a362b4b72","Type":"ContainerDied","Data":"45cca818e240a0080084d27363499bd78019af3256f6cb3467f881433e3be1f6"}
Jul 10 03:33:21.375984 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:21.375947    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:21.376125 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:21.376104    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:33:21.376186 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:21.376175    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls podName:a3a9c521-10bc-4303-beee-302cee1be142 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:22.376158625 +0000 UTC m=+37.890893908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls") pod "dns-default-wszvx" (UID: "a3a9c521-10bc-4303-beee-302cee1be142") : secret "dns-default-metrics-tls" not found
Jul 10 03:33:22.120225 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:22.120194    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:22.122775 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:22.122746    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 03:33:22.122775 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:22.122765    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 03:33:22.122955 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:22.122832    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c4cxw\""
Jul 10 03:33:22.351524 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:22.351471    2571 generic.go:356] "Generic (PLEG): container finished" podID="26fcd562-4c93-4255-9a6d-0c4a362b4b72" containerID="894aa3acea1e016bb9f8449e842ed0f564c5726cb83a526fb2dbf4df182a296d" exitCode=0
Jul 10 03:33:22.351524 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:22.351498    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dhvb5" event={"ID":"26fcd562-4c93-4255-9a6d-0c4a362b4b72","Type":"ContainerDied","Data":"894aa3acea1e016bb9f8449e842ed0f564c5726cb83a526fb2dbf4df182a296d"}
Jul 10 03:33:22.383879 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:22.383848    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:22.384043 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:22.383964    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:33:22.384090 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:22.384048    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls podName:a3a9c521-10bc-4303-beee-302cee1be142 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:24.384029206 +0000 UTC m=+39.898764501 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls") pod "dns-default-wszvx" (UID: "a3a9c521-10bc-4303-beee-302cee1be142") : secret "dns-default-metrics-tls" not found
Jul 10 03:33:23.358516 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:23.358482    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dhvb5" event={"ID":"26fcd562-4c93-4255-9a6d-0c4a362b4b72","Type":"ContainerStarted","Data":"c3f01e6614c27ede4b68df71116eceb17f68932f15012bf234cccd86e750aa5a"}
Jul 10 03:33:23.380113 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:23.380044    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dhvb5" podStartSLOduration=12.455438453 podStartE2EDuration="38.380022544s" totalImagesPullingTime="25.924584091s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:23.379499048 +0000 UTC m=+38.894234352" watchObservedRunningTime="2026-07-10 03:33:23.380022544 +0000 UTC m=+38.894757848"
Jul 10 03:33:24.362321 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:24.362131    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hdnl8" event={"ID":"86afb721-e30b-48ed-b932-7ceca0182679","Type":"ContainerStarted","Data":"5c62d296e2e4d24c02200d3ff9ffca6047325cc85485d489f589ac8152dca6cf"}
Jul 10 03:33:24.378967 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:24.378913    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hdnl8" podStartSLOduration=1.659929312 podStartE2EDuration="4.378895874s" totalImagesPullingTime="2.718966562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:24.377660024 +0000 UTC m=+39.892395327" watchObservedRunningTime="2026-07-10 03:33:24.378895874 +0000 UTC m=+39.893631176"
Jul 10 03:33:24.399778 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:24.399742    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:24.399940 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:24.399859    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:33:24.399940 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:24.399912    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls podName:a3a9c521-10bc-4303-beee-302cee1be142 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:28.399898142 +0000 UTC m=+43.914633424 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls") pod "dns-default-wszvx" (UID: "a3a9c521-10bc-4303-beee-302cee1be142") : secret "dns-default-metrics-tls" not found
Jul 10 03:33:28.431011 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:28.430966    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:28.431554 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:28.431125    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:33:28.431554 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:28.431194    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls podName:a3a9c521-10bc-4303-beee-302cee1be142 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:36.431178082 +0000 UTC m=+51.945913364 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls") pod "dns-default-wszvx" (UID: "a3a9c521-10bc-4303-beee-302cee1be142") : secret "dns-default-metrics-tls" not found
Jul 10 03:33:36.487236 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:36.487194    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:36.487854 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:36.487320    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:33:36.487854 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:36.487372    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls podName:a3a9c521-10bc-4303-beee-302cee1be142 nodeName:}" failed. No retries permitted until 2026-07-10 03:33:52.487358403 +0000 UTC m=+68.002093685 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls") pod "dns-default-wszvx" (UID: "a3a9c521-10bc-4303-beee-302cee1be142") : secret "dns-default-metrics-tls" not found
Jul 10 03:33:45.198252 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.198221    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"]
Jul 10 03:33:45.200666 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.200649    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:33:45.202936 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.202914    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 10 03:33:45.203746 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.203723    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 10 03:33:45.204029 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.204009    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 10 03:33:45.204134 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.204123    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 10 03:33:45.208725 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.208704    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"]
Jul 10 03:33:45.217291 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.217266    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"]
Jul 10 03:33:45.218859 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.218844    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.221326 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.221306    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 10 03:33:45.221437 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.221314    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 10 03:33:45.221437 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.221364    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 10 03:33:45.221574 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.221513    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 10 03:33:45.231653 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.231629    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"]
Jul 10 03:33:45.352388 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.352347    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/027627e2-0a96-4507-a3be-03e811133cd6-hub\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.352388 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.352384    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/027627e2-0a96-4507-a3be-03e811133cd6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.352643 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.352408    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b43ae079-05df-49da-9590-31c6378da7a3-tmp\") pod \"klusterlet-addon-workmgr-665bd4899f-hvhkk\" (UID: \"b43ae079-05df-49da-9590-31c6378da7a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:33:45.352643 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.352424    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr2sh\" (UniqueName: \"kubernetes.io/projected/b43ae079-05df-49da-9590-31c6378da7a3-kube-api-access-wr2sh\") pod \"klusterlet-addon-workmgr-665bd4899f-hvhkk\" (UID: \"b43ae079-05df-49da-9590-31c6378da7a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:33:45.352643 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.352488    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b43ae079-05df-49da-9590-31c6378da7a3-klusterlet-config\") pod \"klusterlet-addon-workmgr-665bd4899f-hvhkk\" (UID: \"b43ae079-05df-49da-9590-31c6378da7a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:33:45.352643 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.352532    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/027627e2-0a96-4507-a3be-03e811133cd6-ca\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.352643 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.352606    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/027627e2-0a96-4507-a3be-03e811133cd6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.352643 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.352641    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/027627e2-0a96-4507-a3be-03e811133cd6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.352843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.352658    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbp7s\" (UniqueName: \"kubernetes.io/projected/027627e2-0a96-4507-a3be-03e811133cd6-kube-api-access-bbp7s\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.453355 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.453240    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/027627e2-0a96-4507-a3be-03e811133cd6-hub\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.453355 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.453301    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/027627e2-0a96-4507-a3be-03e811133cd6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.453355 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.453328    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b43ae079-05df-49da-9590-31c6378da7a3-tmp\") pod \"klusterlet-addon-workmgr-665bd4899f-hvhkk\" (UID: \"b43ae079-05df-49da-9590-31c6378da7a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:33:45.453355 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.453351    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr2sh\" (UniqueName: \"kubernetes.io/projected/b43ae079-05df-49da-9590-31c6378da7a3-kube-api-access-wr2sh\") pod \"klusterlet-addon-workmgr-665bd4899f-hvhkk\" (UID: \"b43ae079-05df-49da-9590-31c6378da7a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:33:45.453720 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.453386    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b43ae079-05df-49da-9590-31c6378da7a3-klusterlet-config\") pod \"klusterlet-addon-workmgr-665bd4899f-hvhkk\" (UID: \"b43ae079-05df-49da-9590-31c6378da7a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:33:45.453720 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.453452    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/027627e2-0a96-4507-a3be-03e811133cd6-ca\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.453720 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.453548    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/027627e2-0a96-4507-a3be-03e811133cd6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.453720 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.453609    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/027627e2-0a96-4507-a3be-03e811133cd6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.453720 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.453633    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbp7s\" (UniqueName: \"kubernetes.io/projected/027627e2-0a96-4507-a3be-03e811133cd6-kube-api-access-bbp7s\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.453959 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.453849    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b43ae079-05df-49da-9590-31c6378da7a3-tmp\") pod \"klusterlet-addon-workmgr-665bd4899f-hvhkk\" (UID: \"b43ae079-05df-49da-9590-31c6378da7a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:33:45.454210 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.454183    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/027627e2-0a96-4507-a3be-03e811133cd6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.457481 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.457435    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/027627e2-0a96-4507-a3be-03e811133cd6-hub\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.457619 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.457498    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/027627e2-0a96-4507-a3be-03e811133cd6-ca\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.457619 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.457548    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/027627e2-0a96-4507-a3be-03e811133cd6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.457733 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.457644    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b43ae079-05df-49da-9590-31c6378da7a3-klusterlet-config\") pod \"klusterlet-addon-workmgr-665bd4899f-hvhkk\" (UID: \"b43ae079-05df-49da-9590-31c6378da7a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:33:45.457877 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.457858    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/027627e2-0a96-4507-a3be-03e811133cd6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.460703 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.460679    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr2sh\" (UniqueName: \"kubernetes.io/projected/b43ae079-05df-49da-9590-31c6378da7a3-kube-api-access-wr2sh\") pod \"klusterlet-addon-workmgr-665bd4899f-hvhkk\" (UID: \"b43ae079-05df-49da-9590-31c6378da7a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:33:45.461046 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.461025    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbp7s\" (UniqueName: \"kubernetes.io/projected/027627e2-0a96-4507-a3be-03e811133cd6-kube-api-access-bbp7s\") pod \"cluster-proxy-proxy-agent-85bff74d6-zvhrp\" (UID: \"027627e2-0a96-4507-a3be-03e811133cd6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.509766 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.509727    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:33:45.543946 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.543912    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:33:45.639936 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.639828    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"]
Jul 10 03:33:45.644343 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:33:45.644315    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43ae079_05df_49da_9590_31c6378da7a3.slice/crio-6c88c0bc8df76880dc21a498027152d259f0591e51530cb191edb1dba28f7106 WatchSource:0}: Error finding container 6c88c0bc8df76880dc21a498027152d259f0591e51530cb191edb1dba28f7106: Status 404 returned error can't find the container with id 6c88c0bc8df76880dc21a498027152d259f0591e51530cb191edb1dba28f7106
Jul 10 03:33:45.685482 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:45.685427    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"]
Jul 10 03:33:45.689593 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:33:45.689565    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod027627e2_0a96_4507_a3be_03e811133cd6.slice/crio-934119b22f18b0b08995412cca77ad6bdf4b185a53771bab68cbc2cc8a42341b WatchSource:0}: Error finding container 934119b22f18b0b08995412cca77ad6bdf4b185a53771bab68cbc2cc8a42341b: Status 404 returned error can't find the container with id 934119b22f18b0b08995412cca77ad6bdf4b185a53771bab68cbc2cc8a42341b
Jul 10 03:33:46.336486 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:46.336442    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gxvf2"
Jul 10 03:33:46.417833 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:46.417799    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk" event={"ID":"b43ae079-05df-49da-9590-31c6378da7a3","Type":"ContainerStarted","Data":"6c88c0bc8df76880dc21a498027152d259f0591e51530cb191edb1dba28f7106"}
Jul 10 03:33:46.419025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:46.418993    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp" event={"ID":"027627e2-0a96-4507-a3be-03e811133cd6","Type":"ContainerStarted","Data":"934119b22f18b0b08995412cca77ad6bdf4b185a53771bab68cbc2cc8a42341b"}
Jul 10 03:33:50.431740 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:50.431699    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp" event={"ID":"027627e2-0a96-4507-a3be-03e811133cd6","Type":"ContainerStarted","Data":"c2830cfa48a9608100cf36657289eae496ec6c53ac157b6052aa844fed86c660"}
Jul 10 03:33:50.432149 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:50.431748    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp" event={"ID":"027627e2-0a96-4507-a3be-03e811133cd6","Type":"ContainerStarted","Data":"ae8a19fa786954c988b82845116d105c73aadb452bc6d6bf6ca00e2969262468"}
Jul 10 03:33:50.432149 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:50.431766    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp" event={"ID":"027627e2-0a96-4507-a3be-03e811133cd6","Type":"ContainerStarted","Data":"823c50ae96cbb3e08c16f2c3885d00016d1848f706d8fa5b658ae69c659a8a48"}
Jul 10 03:33:50.449033 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:50.448930    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp" podStartSLOduration=0.954324901 podStartE2EDuration="5.448908258s" totalImagesPullingTime="4.494583357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:33:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:50.447860975 +0000 UTC m=+65.962596300" watchObservedRunningTime="2026-07-10 03:33:50.448908258 +0000 UTC m=+65.963643563"
Jul 10 03:33:51.802158 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:51.802119    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:33:51.802656 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:51.802193    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:51.804753 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:51.804730    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 03:33:51.804753 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:51.804748    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 03:33:51.812719 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:51.812700    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 03:33:51.812819 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:51.812775    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs podName:26778894-1f27-4bcc-b3de-5ae0ac3aca4c nodeName:}" failed. No retries permitted until 2026-07-10 03:34:55.812750377 +0000 UTC m=+131.327485660 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs") pod "network-metrics-daemon-j6vpn" (UID: "26778894-1f27-4bcc-b3de-5ae0ac3aca4c") : secret "metrics-daemon-secret" not found
Jul 10 03:33:51.814916 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:51.814893    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/859d3328-63da-49c4-a871-2b45a4960764-original-pull-secret\") pod \"global-pull-secret-syncer-vlr5v\" (UID: \"859d3328-63da-49c4-a871-2b45a4960764\") " pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:51.903416 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:51.903381    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzwqc\" (UniqueName: \"kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc\") pod \"network-check-target-zkhkn\" (UID: \"ef22232c-35eb-4554-8518-ea64524eefa1\") " pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:51.906264 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:51.906236    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 03:33:51.916066 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:51.916044    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 03:33:51.927336 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:51.927310    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzwqc\" (UniqueName: \"kubernetes.io/projected/ef22232c-35eb-4554-8518-ea64524eefa1-kube-api-access-jzwqc\") pod \"network-check-target-zkhkn\" (UID: \"ef22232c-35eb-4554-8518-ea64524eefa1\") " pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:52.034604 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:52.034561    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlr5v"
Jul 10 03:33:52.135506 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:52.135470    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c4cxw\""
Jul 10 03:33:52.143338 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:52.143305    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:52.194056 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:52.193988    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vlr5v"]
Jul 10 03:33:52.197209 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:33:52.197180    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod859d3328_63da_49c4_a871_2b45a4960764.slice/crio-92c3ef70f7de1e9d0f16e8072a27190f8a672462a2bcdf255e965ba261c46f96 WatchSource:0}: Error finding container 92c3ef70f7de1e9d0f16e8072a27190f8a672462a2bcdf255e965ba261c46f96: Status 404 returned error can't find the container with id 92c3ef70f7de1e9d0f16e8072a27190f8a672462a2bcdf255e965ba261c46f96
Jul 10 03:33:52.296094 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:52.296062    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zkhkn"]
Jul 10 03:33:52.299448 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:33:52.299427    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef22232c_35eb_4554_8518_ea64524eefa1.slice/crio-eac97baf58e32832b699ae8fbce106bdea7e275432dcd8f230f2a5c5067038fe WatchSource:0}: Error finding container eac97baf58e32832b699ae8fbce106bdea7e275432dcd8f230f2a5c5067038fe: Status 404 returned error can't find the container with id eac97baf58e32832b699ae8fbce106bdea7e275432dcd8f230f2a5c5067038fe
Jul 10 03:33:52.438148 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:52.438106    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vlr5v" event={"ID":"859d3328-63da-49c4-a871-2b45a4960764","Type":"ContainerStarted","Data":"92c3ef70f7de1e9d0f16e8072a27190f8a672462a2bcdf255e965ba261c46f96"}
Jul 10 03:33:52.439369 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:52.439336    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zkhkn" event={"ID":"ef22232c-35eb-4554-8518-ea64524eefa1","Type":"ContainerStarted","Data":"eac97baf58e32832b699ae8fbce106bdea7e275432dcd8f230f2a5c5067038fe"}
Jul 10 03:33:52.513025 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:52.512989    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:33:52.513214 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:52.513192    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:33:52.513287 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:33:52.513274    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls podName:a3a9c521-10bc-4303-beee-302cee1be142 nodeName:}" failed. No retries permitted until 2026-07-10 03:34:24.513252698 +0000 UTC m=+100.027987993 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls") pod "dns-default-wszvx" (UID: "a3a9c521-10bc-4303-beee-302cee1be142") : secret "dns-default-metrics-tls" not found
Jul 10 03:33:54.447753 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:54.447708    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk" event={"ID":"b43ae079-05df-49da-9590-31c6378da7a3","Type":"ContainerStarted","Data":"047f49a519ec3e4d5b02e0b37e268eeaff31e38c1323545246c10c652d9c5b64"}
Jul 10 03:33:54.448532 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:54.448505    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:33:54.450573 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:54.450550    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:33:54.463831 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:54.463762    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk" podStartSLOduration=1.6633303289999999 podStartE2EDuration="9.463743921s" totalImagesPullingTime="7.800413592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:33:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:54.462535956 +0000 UTC m=+69.977271261" watchObservedRunningTime="2026-07-10 03:33:54.463743921 +0000 UTC m=+69.978479225"
Jul 10 03:33:58.461742 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:58.461703    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zkhkn" event={"ID":"ef22232c-35eb-4554-8518-ea64524eefa1","Type":"ContainerStarted","Data":"0024af57d0b96e6d9eaedb8a126cfbd31a9b33158eb09820b0b39514b60e7aa0"}
Jul 10 03:33:58.462205 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:58.461789    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:33:58.463185 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:58.463153    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vlr5v" event={"ID":"859d3328-63da-49c4-a871-2b45a4960764","Type":"ContainerStarted","Data":"c6a16a54d11d6d1e82dadd40bd1f382e2efd7b0070ccb0fbb74d54180260660a"}
Jul 10 03:33:58.477622 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:58.477570    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zkhkn" podStartSLOduration=67.94600231 podStartE2EDuration="1m13.477554433s" totalImagesPullingTime="5.531552123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:58.47718471 +0000 UTC m=+73.991920014" watchObservedRunningTime="2026-07-10 03:33:58.477554433 +0000 UTC m=+73.992289736"
Jul 10 03:33:58.490944 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:33:58.490890    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vlr5v" podStartSLOduration=67.845912092 podStartE2EDuration="1m13.49087496s" totalImagesPullingTime="5.644962868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:33:58.490529311 +0000 UTC m=+74.005264615" watchObservedRunningTime="2026-07-10 03:33:58.49087496 +0000 UTC m=+74.005610263"
Jul 10 03:34:24.547275 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:34:24.547240    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:34:24.547715 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:34:24.547393    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:34:24.547715 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:34:24.547488    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls podName:a3a9c521-10bc-4303-beee-302cee1be142 nodeName:}" failed. No retries permitted until 2026-07-10 03:35:28.547452362 +0000 UTC m=+164.062187644 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls") pod "dns-default-wszvx" (UID: "a3a9c521-10bc-4303-beee-302cee1be142") : secret "dns-default-metrics-tls" not found
Jul 10 03:34:29.470065 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:34:29.470032    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zkhkn"
Jul 10 03:34:55.868200 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:34:55.868142    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:34:55.868735 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:34:55.868277    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 03:34:55.868735 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:34:55.868342    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs podName:26778894-1f27-4bcc-b3de-5ae0ac3aca4c nodeName:}" failed. No retries permitted until 2026-07-10 03:36:57.868327482 +0000 UTC m=+253.383062766 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs") pod "network-metrics-daemon-j6vpn" (UID: "26778894-1f27-4bcc-b3de-5ae0ac3aca4c") : secret "metrics-daemon-secret" not found
Jul 10 03:35:23.692881 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:35:23.692832    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-wszvx" podUID="a3a9c521-10bc-4303-beee-302cee1be142"
Jul 10 03:35:23.707188 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:35:23.707160    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wszvx"
Jul 10 03:35:24.139883 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:35:24.139777    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-j6vpn" podUID="26778894-1f27-4bcc-b3de-5ae0ac3aca4c"
Jul 10 03:35:28.606983 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:35:28.606924    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:35:28.607512 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:35:28.607059    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 03:35:28.607512 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:35:28.607118    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls podName:a3a9c521-10bc-4303-beee-302cee1be142 nodeName:}" failed. No retries permitted until 2026-07-10 03:37:30.607102009 +0000 UTC m=+286.121837313 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls") pod "dns-default-wszvx" (UID: "a3a9c521-10bc-4303-beee-302cee1be142") : secret "dns-default-metrics-tls" not found
Jul 10 03:35:38.120412 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:35:38.120325    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:35:53.793609 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:35:53.793525    2571 generic.go:356] "Generic (PLEG): container finished" podID="b43ae079-05df-49da-9590-31c6378da7a3" containerID="047f49a519ec3e4d5b02e0b37e268eeaff31e38c1323545246c10c652d9c5b64" exitCode=1
Jul 10 03:35:53.793609 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:35:53.793597    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk" event={"ID":"b43ae079-05df-49da-9590-31c6378da7a3","Type":"ContainerDied","Data":"047f49a519ec3e4d5b02e0b37e268eeaff31e38c1323545246c10c652d9c5b64"}
Jul 10 03:35:53.794004 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:35:53.793954    2571 scope.go:117] "RemoveContainer" containerID="047f49a519ec3e4d5b02e0b37e268eeaff31e38c1323545246c10c652d9c5b64"
Jul 10 03:35:54.449007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:35:54.448971    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:35:54.798411 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:35:54.798325    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk" event={"ID":"b43ae079-05df-49da-9590-31c6378da7a3","Type":"ContainerStarted","Data":"9d5fe102ad33c0a561ad49403d2a8103fff02a8b05520dc365b40bd4ab7d2c1d"}
Jul 10 03:35:54.798776 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:35:54.798555    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:35:54.799184 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:35:54.799169    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-665bd4899f-hvhkk"
Jul 10 03:36:29.295983 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.295950    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bqmg8"]
Jul 10 03:36:29.298040 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.298005    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:29.300214 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.300191    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 03:36:29.300318 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.300195    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lwzwp\""
Jul 10 03:36:29.300318 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.300248    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 03:36:29.300918 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.300902    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 03:36:29.301018 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.300945    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 03:36:29.307744 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.307720    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bqmg8"]
Jul 10 03:36:29.422905 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.422855    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwrtb\" (UniqueName: \"kubernetes.io/projected/9df437ac-228c-4177-84cd-effed0e43caf-kube-api-access-hwrtb\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:29.422905 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.422925    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9df437ac-228c-4177-84cd-effed0e43caf-data-volume\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:29.423125 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.422965    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:29.423125 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.422984    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:29.423125 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.423062    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9df437ac-228c-4177-84cd-effed0e43caf-crio-socket\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:29.523571 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.523527    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:29.523571 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.523570    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:29.523806 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.523614    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9df437ac-228c-4177-84cd-effed0e43caf-crio-socket\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:29.523806 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.523639    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwrtb\" (UniqueName: \"kubernetes.io/projected/9df437ac-228c-4177-84cd-effed0e43caf-kube-api-access-hwrtb\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:29.523806 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:29.523659    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 03:36:29.523806 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.523668    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9df437ac-228c-4177-84cd-effed0e43caf-data-volume\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:29.523806 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.523736    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9df437ac-228c-4177-84cd-effed0e43caf-crio-socket\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:29.523806 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:29.523747    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm podName:9df437ac-228c-4177-84cd-effed0e43caf nodeName:}" failed. No retries permitted until 2026-07-10 03:36:30.023726772 +0000 UTC m=+225.538462054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqmg8" (UID: "9df437ac-228c-4177-84cd-effed0e43caf") : configmap "kube-rbac-proxy" not found
Jul 10 03:36:29.523806 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:29.523801    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:29.524060 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:29.523879    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls podName:9df437ac-228c-4177-84cd-effed0e43caf nodeName:}" failed. No retries permitted until 2026-07-10 03:36:30.02386166 +0000 UTC m=+225.538596954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqmg8" (UID: "9df437ac-228c-4177-84cd-effed0e43caf") : secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:29.524060 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.523938    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9df437ac-228c-4177-84cd-effed0e43caf-data-volume\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:29.531885 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:29.531860    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwrtb\" (UniqueName: \"kubernetes.io/projected/9df437ac-228c-4177-84cd-effed0e43caf-kube-api-access-hwrtb\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:30.027986 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:30.027949    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:30.027986 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:30.027987    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:30.028193 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:30.028071    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 03:36:30.028193 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:30.028095    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:30.028193 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:30.028141    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm podName:9df437ac-228c-4177-84cd-effed0e43caf nodeName:}" failed. No retries permitted until 2026-07-10 03:36:31.028117668 +0000 UTC m=+226.542852953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqmg8" (UID: "9df437ac-228c-4177-84cd-effed0e43caf") : configmap "kube-rbac-proxy" not found
Jul 10 03:36:30.028193 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:30.028155    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls podName:9df437ac-228c-4177-84cd-effed0e43caf nodeName:}" failed. No retries permitted until 2026-07-10 03:36:31.028149349 +0000 UTC m=+226.542884633 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqmg8" (UID: "9df437ac-228c-4177-84cd-effed0e43caf") : secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:31.035424 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:31.035387    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:31.035424 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:31.035425    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:31.035858 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:31.035524    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 03:36:31.035858 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:31.035564    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:31.035858 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:31.035586    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm podName:9df437ac-228c-4177-84cd-effed0e43caf nodeName:}" failed. No retries permitted until 2026-07-10 03:36:33.035570551 +0000 UTC m=+228.550305832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqmg8" (UID: "9df437ac-228c-4177-84cd-effed0e43caf") : configmap "kube-rbac-proxy" not found
Jul 10 03:36:31.035858 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:31.035606    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls podName:9df437ac-228c-4177-84cd-effed0e43caf nodeName:}" failed. No retries permitted until 2026-07-10 03:36:33.035594934 +0000 UTC m=+228.550330215 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqmg8" (UID: "9df437ac-228c-4177-84cd-effed0e43caf") : secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:31.879789 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:31.879762    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9p7cz_4b1d7593-1d0c-4166-8dc8-733dc6c4f55c/dns-node-resolver/0.log"
Jul 10 03:36:33.050723 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:33.050684    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:33.050723 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:33.050723    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:33.051184 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:33.050807    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 03:36:33.051184 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:33.050855    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:33.051184 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:33.050869    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm podName:9df437ac-228c-4177-84cd-effed0e43caf nodeName:}" failed. No retries permitted until 2026-07-10 03:36:37.050854268 +0000 UTC m=+232.565589550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqmg8" (UID: "9df437ac-228c-4177-84cd-effed0e43caf") : configmap "kube-rbac-proxy" not found
Jul 10 03:36:33.051184 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:33.050893    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls podName:9df437ac-228c-4177-84cd-effed0e43caf nodeName:}" failed. No retries permitted until 2026-07-10 03:36:37.050882522 +0000 UTC m=+232.565617803 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqmg8" (UID: "9df437ac-228c-4177-84cd-effed0e43caf") : secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:33.279770 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:33.279742    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xnfdx_bfe8cc34-69bf-4d9e-a945-b1c867b6111a/node-ca/0.log"
Jul 10 03:36:33.679503 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:33.679470    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hdnl8_86afb721-e30b-48ed-b932-7ceca0182679/serve-healthcheck-canary/0.log"
Jul 10 03:36:37.083366 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:37.083311    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:37.083366 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:37.083370    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:37.083847 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:37.083435    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 03:36:37.083847 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:37.083531    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm podName:9df437ac-228c-4177-84cd-effed0e43caf nodeName:}" failed. No retries permitted until 2026-07-10 03:36:45.083515326 +0000 UTC m=+240.598250607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqmg8" (UID: "9df437ac-228c-4177-84cd-effed0e43caf") : configmap "kube-rbac-proxy" not found
Jul 10 03:36:37.083847 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:37.083544    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:37.083847 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:36:37.083601    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls podName:9df437ac-228c-4177-84cd-effed0e43caf nodeName:}" failed. No retries permitted until 2026-07-10 03:36:45.083588214 +0000 UTC m=+240.598323496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqmg8" (UID: "9df437ac-228c-4177-84cd-effed0e43caf") : secret "insights-runtime-extractor-tls" not found
Jul 10 03:36:45.139386 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:45.139345    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:45.139386 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:45.139387    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:45.139869 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:45.139849    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9df437ac-228c-4177-84cd-effed0e43caf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:45.141502 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:45.141485    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9df437ac-228c-4177-84cd-effed0e43caf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqmg8\" (UID: \"9df437ac-228c-4177-84cd-effed0e43caf\") " pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:45.209611 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:45.209584    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lwzwp\""
Jul 10 03:36:45.218318 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:45.218283    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bqmg8"
Jul 10 03:36:45.340455 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:45.340426    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bqmg8"]
Jul 10 03:36:45.343524 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:36:45.343496    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df437ac_228c_4177_84cd_effed0e43caf.slice/crio-a986b2d056f1b465c63c02e27570617250a9d614c4892fd783688776909f2045 WatchSource:0}: Error finding container a986b2d056f1b465c63c02e27570617250a9d614c4892fd783688776909f2045: Status 404 returned error can't find the container with id a986b2d056f1b465c63c02e27570617250a9d614c4892fd783688776909f2045
Jul 10 03:36:45.938399 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:45.938356    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqmg8" event={"ID":"9df437ac-228c-4177-84cd-effed0e43caf","Type":"ContainerStarted","Data":"a986b2d056f1b465c63c02e27570617250a9d614c4892fd783688776909f2045"}
Jul 10 03:36:48.949863 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:48.949824    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqmg8" event={"ID":"9df437ac-228c-4177-84cd-effed0e43caf","Type":"ContainerStarted","Data":"3f6bf8b04eaa31dbf7c395d05ac50a38e4516c6218ab44495236ef995ca83441"}
Jul 10 03:36:49.956013 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:49.955978    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqmg8" event={"ID":"9df437ac-228c-4177-84cd-effed0e43caf","Type":"ContainerStarted","Data":"285288a78024617c03740936290b9b2e1227f90bdacff35eca7cf811c438c45b"}
Jul 10 03:36:50.457142 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.457107    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-667654d5f7-qvstj"]
Jul 10 03:36:50.459050 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.459028    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.461612 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.461572    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 10 03:36:50.461612 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.461595    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 10 03:36:50.461826 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.461749    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tqxc2\""
Jul 10 03:36:50.462378 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.462348    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 10 03:36:50.466442 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.466423    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 10 03:36:50.472510 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.472479    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-667654d5f7-qvstj"]
Jul 10 03:36:50.476422 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.476395    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0518043-17c4-426e-9bc7-3a49f827589f-ca-trust-extracted\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.476582 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.476432    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdlkm\" (UniqueName: \"kubernetes.io/projected/a0518043-17c4-426e-9bc7-3a49f827589f-kube-api-access-cdlkm\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.476582 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.476482    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0518043-17c4-426e-9bc7-3a49f827589f-registry-certificates\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.476582 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.476554    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0518043-17c4-426e-9bc7-3a49f827589f-installation-pull-secrets\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.476730 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.476597    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0518043-17c4-426e-9bc7-3a49f827589f-trusted-ca\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.476730 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.476630    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0518043-17c4-426e-9bc7-3a49f827589f-image-registry-private-configuration\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.476730 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.476714    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0518043-17c4-426e-9bc7-3a49f827589f-bound-sa-token\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.476853 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.476757    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0518043-17c4-426e-9bc7-3a49f827589f-registry-tls\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.577695 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.577660    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0518043-17c4-426e-9bc7-3a49f827589f-ca-trust-extracted\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.577695 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.577699    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdlkm\" (UniqueName: \"kubernetes.io/projected/a0518043-17c4-426e-9bc7-3a49f827589f-kube-api-access-cdlkm\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.577939 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.577723    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0518043-17c4-426e-9bc7-3a49f827589f-registry-certificates\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.577939 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.577756    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0518043-17c4-426e-9bc7-3a49f827589f-installation-pull-secrets\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.577939 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.577793    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0518043-17c4-426e-9bc7-3a49f827589f-trusted-ca\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.577939 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.577825    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0518043-17c4-426e-9bc7-3a49f827589f-image-registry-private-configuration\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.577939 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.577866    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0518043-17c4-426e-9bc7-3a49f827589f-bound-sa-token\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.577939 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.577929    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0518043-17c4-426e-9bc7-3a49f827589f-registry-tls\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.578226 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.578116    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a0518043-17c4-426e-9bc7-3a49f827589f-ca-trust-extracted\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.578801 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.578762    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a0518043-17c4-426e-9bc7-3a49f827589f-registry-certificates\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.579031 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.579005    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0518043-17c4-426e-9bc7-3a49f827589f-trusted-ca\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.580724 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.580688    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a0518043-17c4-426e-9bc7-3a49f827589f-installation-pull-secrets\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.580815 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.580753    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a0518043-17c4-426e-9bc7-3a49f827589f-image-registry-private-configuration\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.580980 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.580955    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a0518043-17c4-426e-9bc7-3a49f827589f-registry-tls\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.585667 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.585631    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0518043-17c4-426e-9bc7-3a49f827589f-bound-sa-token\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.586447 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.586427    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdlkm\" (UniqueName: \"kubernetes.io/projected/a0518043-17c4-426e-9bc7-3a49f827589f-kube-api-access-cdlkm\") pod \"image-registry-667654d5f7-qvstj\" (UID: \"a0518043-17c4-426e-9bc7-3a49f827589f\") " pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:50.771118 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:50.771042    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:51.891883 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:51.891850    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-667654d5f7-qvstj"]
Jul 10 03:36:51.894548 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:36:51.894514    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0518043_17c4_426e_9bc7_3a49f827589f.slice/crio-fac40e230bd83ec8999e6809c62231a7e4eb754cd56b7d29cd9d592fc9405d6a WatchSource:0}: Error finding container fac40e230bd83ec8999e6809c62231a7e4eb754cd56b7d29cd9d592fc9405d6a: Status 404 returned error can't find the container with id fac40e230bd83ec8999e6809c62231a7e4eb754cd56b7d29cd9d592fc9405d6a
Jul 10 03:36:51.964074 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:51.964040    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-667654d5f7-qvstj" event={"ID":"a0518043-17c4-426e-9bc7-3a49f827589f","Type":"ContainerStarted","Data":"c321b6756f4f9f132f28252ed429d501fd955fa4fbc62c7dbf691d84789285a1"}
Jul 10 03:36:51.964255 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:51.964095    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-667654d5f7-qvstj" event={"ID":"a0518043-17c4-426e-9bc7-3a49f827589f","Type":"ContainerStarted","Data":"fac40e230bd83ec8999e6809c62231a7e4eb754cd56b7d29cd9d592fc9405d6a"}
Jul 10 03:36:51.964255 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:51.964124    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:36:51.966074 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:51.966046    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqmg8" event={"ID":"9df437ac-228c-4177-84cd-effed0e43caf","Type":"ContainerStarted","Data":"b56ec25acd3c74c853ae428b7aa22b837c0830fd21465726462b791b2a8b260c"}
Jul 10 03:36:51.981455 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:51.981411    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-667654d5f7-qvstj" podStartSLOduration=1.981395609 podStartE2EDuration="1.981395609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:36:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:36:51.980149018 +0000 UTC m=+247.494884318" watchObservedRunningTime="2026-07-10 03:36:51.981395609 +0000 UTC m=+247.496130911"
Jul 10 03:36:51.996755 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:51.996710    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bqmg8" podStartSLOduration=16.639666621 podStartE2EDuration="22.996688752s" totalImagesPullingTime="6.357022131s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:36:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:36:51.996177366 +0000 UTC m=+247.510912660" watchObservedRunningTime="2026-07-10 03:36:51.996688752 +0000 UTC m=+247.511424056"
Jul 10 03:36:57.934760 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:57.934722    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:36:57.937039 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:57.937014    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26778894-1f27-4bcc-b3de-5ae0ac3aca4c-metrics-certs\") pod \"network-metrics-daemon-j6vpn\" (UID: \"26778894-1f27-4bcc-b3de-5ae0ac3aca4c\") " pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:36:58.223335 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:58.223256    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sg5lf\""
Jul 10 03:36:58.232072 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:58.232041    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6vpn"
Jul 10 03:36:58.358852 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:58.358823    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j6vpn"]
Jul 10 03:36:58.362455 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:36:58.362430    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26778894_1f27_4bcc_b3de_5ae0ac3aca4c.slice/crio-56e579407b7c62dc6b6efd4e03b2df2cf079c0d3d823e5d4b15a52da00ae1379 WatchSource:0}: Error finding container 56e579407b7c62dc6b6efd4e03b2df2cf079c0d3d823e5d4b15a52da00ae1379: Status 404 returned error can't find the container with id 56e579407b7c62dc6b6efd4e03b2df2cf079c0d3d823e5d4b15a52da00ae1379
Jul 10 03:36:58.989795 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:58.989760    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j6vpn" event={"ID":"26778894-1f27-4bcc-b3de-5ae0ac3aca4c","Type":"ContainerStarted","Data":"56e579407b7c62dc6b6efd4e03b2df2cf079c0d3d823e5d4b15a52da00ae1379"}
Jul 10 03:36:59.994969 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:59.994934    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j6vpn" event={"ID":"26778894-1f27-4bcc-b3de-5ae0ac3aca4c","Type":"ContainerStarted","Data":"f7cbfaaf2335efe07a631fcc9701d6a707a0365ba9f5414639e811c17e8b9624"}
Jul 10 03:36:59.995365 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:36:59.995018    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j6vpn" event={"ID":"26778894-1f27-4bcc-b3de-5ae0ac3aca4c","Type":"ContainerStarted","Data":"84b7e5915daab568f82b6e6530321407670b896d66e2ffc050a901f216e0f63a"}
Jul 10 03:37:00.008817 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:00.008768    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-j6vpn" podStartSLOduration=254.057747971 podStartE2EDuration="4m15.008752558s" totalImagesPullingTime="951.004587ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:00.007453259 +0000 UTC m=+255.522188563" watchObservedRunningTime="2026-07-10 03:37:00.008752558 +0000 UTC m=+255.523487911"
Jul 10 03:37:10.776240 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:10.776156    2571 patch_prober.go:28] interesting pod/image-registry-667654d5f7-qvstj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 10 03:37:10.776240 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:10.776206    2571 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-667654d5f7-qvstj" podUID="a0518043-17c4-426e-9bc7-3a49f827589f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 03:37:12.974045 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:12.974009    2571 patch_prober.go:28] interesting pod/image-registry-667654d5f7-qvstj container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 10 03:37:12.974445 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:12.974060    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-667654d5f7-qvstj" podUID="a0518043-17c4-426e-9bc7-3a49f827589f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 03:37:14.220526 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.220494    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rxbc8"]
Jul 10 03:37:14.223509 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.223476    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.226497 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.226447    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 03:37:14.226497 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.226472    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 03:37:14.226497 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.226494    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 03:37:14.226701 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.226497    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 03:37:14.226701 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.226522    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 03:37:14.226701 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.226452    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8h9sp\""
Jul 10 03:37:14.226701 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.226639    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 03:37:14.353671 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.353637    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-accelerators-collector-config\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.353671 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.353678    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f1d8a47-c493-483c-a550-935c74a11a07-sys\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.353879 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.353702    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-textfile\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.353879 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.353725    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-wtmp\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.353879 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.353751    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwr5f\" (UniqueName: \"kubernetes.io/projected/3f1d8a47-c493-483c-a550-935c74a11a07-kube-api-access-nwr5f\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.353879 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.353773    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.353879 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.353796    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-tls\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.353879 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.353857    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f1d8a47-c493-483c-a550-935c74a11a07-metrics-client-ca\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.354051 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.353891    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f1d8a47-c493-483c-a550-935c74a11a07-root\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455285 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455248    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f1d8a47-c493-483c-a550-935c74a11a07-metrics-client-ca\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455285 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455288    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f1d8a47-c493-483c-a550-935c74a11a07-root\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455557 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455330    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-accelerators-collector-config\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455557 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455354    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f1d8a47-c493-483c-a550-935c74a11a07-sys\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455557 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455372    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-textfile\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455557 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455387    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-wtmp\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455557 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455403    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwr5f\" (UniqueName: \"kubernetes.io/projected/3f1d8a47-c493-483c-a550-935c74a11a07-kube-api-access-nwr5f\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455557 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455432    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455557 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455448    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f1d8a47-c493-483c-a550-935c74a11a07-sys\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455557 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455472    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-tls\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455557 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455527    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f1d8a47-c493-483c-a550-935c74a11a07-root\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455980 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455675    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-wtmp\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455980 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:37:14.455678    2571 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 10 03:37:14.455980 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:37:14.455760    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-tls podName:3f1d8a47-c493-483c-a550-935c74a11a07 nodeName:}" failed. No retries permitted until 2026-07-10 03:37:14.955739803 +0000 UTC m=+270.470475084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-tls") pod "node-exporter-rxbc8" (UID: "3f1d8a47-c493-483c-a550-935c74a11a07") : secret "node-exporter-tls" not found
Jul 10 03:37:14.455980 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455867    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-textfile\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.455980 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.455969    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-accelerators-collector-config\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.456187 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.456039    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f1d8a47-c493-483c-a550-935c74a11a07-metrics-client-ca\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.457832 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.457808    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.465329 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.465308    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwr5f\" (UniqueName: \"kubernetes.io/projected/3f1d8a47-c493-483c-a550-935c74a11a07-kube-api-access-nwr5f\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.959360 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.959317    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-tls\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:14.961558 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:14.961539    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f1d8a47-c493-483c-a550-935c74a11a07-node-exporter-tls\") pod \"node-exporter-rxbc8\" (UID: \"3f1d8a47-c493-483c-a550-935c74a11a07\") " pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:15.132061 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:15.132029    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rxbc8"
Jul 10 03:37:15.142817 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:37:15.142772    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f1d8a47_c493_483c_a550_935c74a11a07.slice/crio-c1a47e99e04560a4c3d1171ab2e9c541f38c9e22f53cf8ba93d3948a79102c33 WatchSource:0}: Error finding container c1a47e99e04560a4c3d1171ab2e9c541f38c9e22f53cf8ba93d3948a79102c33: Status 404 returned error can't find the container with id c1a47e99e04560a4c3d1171ab2e9c541f38c9e22f53cf8ba93d3948a79102c33
Jul 10 03:37:16.045782 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:16.045751    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rxbc8" event={"ID":"3f1d8a47-c493-483c-a550-935c74a11a07","Type":"ContainerStarted","Data":"58e1df0a9cbd38b60fbe237c1d3d00a79b426b21708d2a06bb895dbb21239a82"}
Jul 10 03:37:16.046136 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:16.045807    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rxbc8" event={"ID":"3f1d8a47-c493-483c-a550-935c74a11a07","Type":"ContainerStarted","Data":"c1a47e99e04560a4c3d1171ab2e9c541f38c9e22f53cf8ba93d3948a79102c33"}
Jul 10 03:37:17.049949 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:17.049917    2571 generic.go:356] "Generic (PLEG): container finished" podID="3f1d8a47-c493-483c-a550-935c74a11a07" containerID="58e1df0a9cbd38b60fbe237c1d3d00a79b426b21708d2a06bb895dbb21239a82" exitCode=0
Jul 10 03:37:17.050331 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:17.049981    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rxbc8" event={"ID":"3f1d8a47-c493-483c-a550-935c74a11a07","Type":"ContainerDied","Data":"58e1df0a9cbd38b60fbe237c1d3d00a79b426b21708d2a06bb895dbb21239a82"}
Jul 10 03:37:18.055159 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:18.055125    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rxbc8" event={"ID":"3f1d8a47-c493-483c-a550-935c74a11a07","Type":"ContainerStarted","Data":"5b5ec898f1218926bb891892182ece18adfe0e50c18d741fd4405c38aa4e82df"}
Jul 10 03:37:18.055592 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:18.055178    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rxbc8" event={"ID":"3f1d8a47-c493-483c-a550-935c74a11a07","Type":"ContainerStarted","Data":"a54e7efcedb2dfa69544749f5dafa03b06a61f6cf41ab20d7a2fb08f1a81d879"}
Jul 10 03:37:18.074287 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:18.074225    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rxbc8" podStartSLOduration=3.193124749 podStartE2EDuration="4.07421136s" totalImagesPullingTime="818.966611ms" totalInitContainerRuntime="62.12ms" isStatefulPod=true podCreationTimestamp="2026-07-10 03:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:18.072879439 +0000 UTC m=+273.587614744" watchObservedRunningTime="2026-07-10 03:37:18.07421136 +0000 UTC m=+273.588946667"
Jul 10 03:37:19.401309 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.401275    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7d49776db7-gtg5m"]
Jul 10 03:37:19.404645 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.404622    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.407052 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.407031    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 10 03:37:19.407183 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.407162    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 10 03:37:19.407241 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.407185    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 10 03:37:19.407324 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.407308    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 10 03:37:19.407411 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.407396    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8dch2\""
Jul 10 03:37:19.407687 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.407673    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 10 03:37:19.412560 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.412543    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 10 03:37:19.417842 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.417818    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d49776db7-gtg5m"]
Jul 10 03:37:19.495858 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.495822    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-telemeter-client-tls\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.495858 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.495860    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.496071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.495919    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-serving-certs-ca-bundle\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.496071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.495953    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqzgz\" (UniqueName: \"kubernetes.io/projected/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-kube-api-access-hqzgz\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.496071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.496005    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.496071 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.496052    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-metrics-client-ca\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.496191 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.496074    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-secret-telemeter-client\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.496191 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.496090    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-federate-client-tls\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.597290 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.597250    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.597496 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.597306    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-metrics-client-ca\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.597496 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.597328    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-secret-telemeter-client\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.597496 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.597346    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-federate-client-tls\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.597496 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.597371    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-telemeter-client-tls\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.597496 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.597391    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.597496 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.597455    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-serving-certs-ca-bundle\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.597846 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.597503    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqzgz\" (UniqueName: \"kubernetes.io/projected/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-kube-api-access-hqzgz\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.598208 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.598177    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-serving-certs-ca-bundle\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.598392 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.598369    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-metrics-client-ca\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.598476 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.598439    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.599922 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.599901    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.600198 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.600175    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-telemeter-client-tls\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.600277 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.600263    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-federate-client-tls\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.600351 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.600333    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-secret-telemeter-client\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.605358 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.605332    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqzgz\" (UniqueName: \"kubernetes.io/projected/4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453-kube-api-access-hqzgz\") pod \"telemeter-client-7d49776db7-gtg5m\" (UID: \"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453\") " pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.714480 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.714425    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m"
Jul 10 03:37:19.849284 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:19.849152    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d49776db7-gtg5m"]
Jul 10 03:37:19.851918 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:37:19.851885    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fcc25bf_dcbc_4fd1_bfcc_84c0fae13453.slice/crio-2a2afe033265d9d1b4659117c75a113ef3a65ca9e6132df531105d6cf3d060f7 WatchSource:0}: Error finding container 2a2afe033265d9d1b4659117c75a113ef3a65ca9e6132df531105d6cf3d060f7: Status 404 returned error can't find the container with id 2a2afe033265d9d1b4659117c75a113ef3a65ca9e6132df531105d6cf3d060f7
Jul 10 03:37:20.062441 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:20.062358    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m" event={"ID":"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453","Type":"ContainerStarted","Data":"2a2afe033265d9d1b4659117c75a113ef3a65ca9e6132df531105d6cf3d060f7"}
Jul 10 03:37:20.776971 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:20.776935    2571 patch_prober.go:28] interesting pod/image-registry-667654d5f7-qvstj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 10 03:37:20.777393 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:20.776989    2571 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-667654d5f7-qvstj" podUID="a0518043-17c4-426e-9bc7-3a49f827589f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 03:37:22.975277 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:22.975244    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-667654d5f7-qvstj"
Jul 10 03:37:23.074500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:23.074447    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m" event={"ID":"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453","Type":"ContainerStarted","Data":"3c19b571dd29f816da3ec2ac1e3447ccd4d1e409b3b12fd58b2003188b7656bd"}
Jul 10 03:37:24.079341 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.079262    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m" event={"ID":"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453","Type":"ContainerStarted","Data":"9730febf078c8624f2448e6c8ad3c2587002c2d66a6ff0406864cabfa657570e"}
Jul 10 03:37:24.079341 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.079326    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m" event={"ID":"4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453","Type":"ContainerStarted","Data":"153d56497547a3819452a5a198276b77ac5e395a4ee3aa0451a1e68154e50d57"}
Jul 10 03:37:24.115964 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.115898    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7d49776db7-gtg5m" podStartSLOduration=1.260892342 podStartE2EDuration="5.115873442s" totalImagesPullingTime="3.8549811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:37:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:24.114997009 +0000 UTC m=+279.629732356" watchObservedRunningTime="2026-07-10 03:37:24.115873442 +0000 UTC m=+279.630608747"
Jul 10 03:37:24.910363 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.910312    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dc85db69d-xqt85"]
Jul 10 03:37:24.912234 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.912219    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:24.914640 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.914615    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 10 03:37:24.914770 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.914749    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 10 03:37:24.915743 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.915723    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 10 03:37:24.915857 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.915814    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 10 03:37:24.915925 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.915856    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-j6wc5\""
Jul 10 03:37:24.915925 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.915861    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 10 03:37:24.916125 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.916104    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 10 03:37:24.916230 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.916129    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 10 03:37:24.921749 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.921731    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 10 03:37:24.924607 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:24.924587    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dc85db69d-xqt85"]
Jul 10 03:37:25.045999 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.045964    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4ab6a03-8755-42aa-8b22-4218e473873e-console-oauth-config\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.045999 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.045997    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwq4z\" (UniqueName: \"kubernetes.io/projected/c4ab6a03-8755-42aa-8b22-4218e473873e-kube-api-access-dwq4z\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.046177 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.046062    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4ab6a03-8755-42aa-8b22-4218e473873e-console-serving-cert\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.046177 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.046086    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-oauth-serving-cert\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.046177 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.046147    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-service-ca\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.046177 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.046171    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-trusted-ca-bundle\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.046343 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.046227    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-console-config\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.146805 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.146773    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4ab6a03-8755-42aa-8b22-4218e473873e-console-serving-cert\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.146805 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.146802    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-oauth-serving-cert\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.147335 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.146867    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-service-ca\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.147335 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.147123    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-trusted-ca-bundle\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.147335 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.147184    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-console-config\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.147335 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.147283    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4ab6a03-8755-42aa-8b22-4218e473873e-console-oauth-config\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.147335 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.147314    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwq4z\" (UniqueName: \"kubernetes.io/projected/c4ab6a03-8755-42aa-8b22-4218e473873e-kube-api-access-dwq4z\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.147653 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.147628    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-oauth-serving-cert\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.147702 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.147652    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-service-ca\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.148022 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.148000    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-console-config\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.148147 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.148124    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-trusted-ca-bundle\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.149497 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.149473    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4ab6a03-8755-42aa-8b22-4218e473873e-console-serving-cert\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.149581 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.149548    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4ab6a03-8755-42aa-8b22-4218e473873e-console-oauth-config\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.157149 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.157114    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwq4z\" (UniqueName: \"kubernetes.io/projected/c4ab6a03-8755-42aa-8b22-4218e473873e-kube-api-access-dwq4z\") pod \"console-5dc85db69d-xqt85\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") " pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.221299 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.221267    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:25.367100 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:25.367076    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dc85db69d-xqt85"]
Jul 10 03:37:25.369290 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:37:25.369265    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4ab6a03_8755_42aa_8b22_4218e473873e.slice/crio-536f9789cd04e87807b7401008a5bd0de02acb0b76aba14e7e5cf7348f55c361 WatchSource:0}: Error finding container 536f9789cd04e87807b7401008a5bd0de02acb0b76aba14e7e5cf7348f55c361: Status 404 returned error can't find the container with id 536f9789cd04e87807b7401008a5bd0de02acb0b76aba14e7e5cf7348f55c361
Jul 10 03:37:26.087329 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:26.087288    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc85db69d-xqt85" event={"ID":"c4ab6a03-8755-42aa-8b22-4218e473873e","Type":"ContainerStarted","Data":"536f9789cd04e87807b7401008a5bd0de02acb0b76aba14e7e5cf7348f55c361"}
Jul 10 03:37:26.708008 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:37:26.707958    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-wszvx" podUID="a3a9c521-10bc-4303-beee-302cee1be142"
Jul 10 03:37:27.091117 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:27.091044    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wszvx"
Jul 10 03:37:29.099062 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:29.099029    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc85db69d-xqt85" event={"ID":"c4ab6a03-8755-42aa-8b22-4218e473873e","Type":"ContainerStarted","Data":"2f2d0d6275548114f9cba5071a54355b037886ec6a52e0a66543d584faa291a6"}
Jul 10 03:37:29.118420 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:29.118338    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dc85db69d-xqt85" podStartSLOduration=2.4377902430000002 podStartE2EDuration="5.118303594s" totalImagesPullingTime="2.680513351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:37:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:29.117314059 +0000 UTC m=+284.632049361" watchObservedRunningTime="2026-07-10 03:37:29.118303594 +0000 UTC m=+284.633038899"
Jul 10 03:37:30.694069 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:30.694036    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:37:30.696347 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:30.696328    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a9c521-10bc-4303-beee-302cee1be142-metrics-tls\") pod \"dns-default-wszvx\" (UID: \"a3a9c521-10bc-4303-beee-302cee1be142\") " pod="openshift-dns/dns-default-wszvx"
Jul 10 03:37:30.994108 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:30.994021    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dk2pk\""
Jul 10 03:37:31.001898 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:31.001872    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wszvx"
Jul 10 03:37:31.127005 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:31.126976    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wszvx"]
Jul 10 03:37:31.130930 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:37:31.130898    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a9c521_10bc_4303_beee_302cee1be142.slice/crio-c2687e0cd15bd72687915ad2d986099ba8afaa9f173e2b7fc7d4d09bd0c89d5a WatchSource:0}: Error finding container c2687e0cd15bd72687915ad2d986099ba8afaa9f173e2b7fc7d4d09bd0c89d5a: Status 404 returned error can't find the container with id c2687e0cd15bd72687915ad2d986099ba8afaa9f173e2b7fc7d4d09bd0c89d5a
Jul 10 03:37:32.110578 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:32.110541    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wszvx" event={"ID":"a3a9c521-10bc-4303-beee-302cee1be142","Type":"ContainerStarted","Data":"c2687e0cd15bd72687915ad2d986099ba8afaa9f173e2b7fc7d4d09bd0c89d5a"}
Jul 10 03:37:33.117769 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:33.117713    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wszvx" event={"ID":"a3a9c521-10bc-4303-beee-302cee1be142","Type":"ContainerStarted","Data":"3c0884d76c92576af20b7447a901353f4ebe26553b5ee77e04cef6a94d66dcda"}
Jul 10 03:37:34.123237 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:34.123203    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wszvx" event={"ID":"a3a9c521-10bc-4303-beee-302cee1be142","Type":"ContainerStarted","Data":"6c7091609970acb5aa96e271dc120696239282a9903465c28c7299154291c475"}
Jul 10 03:37:34.123660 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:34.123299    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wszvx"
Jul 10 03:37:34.143092 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:34.143039    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wszvx" podStartSLOduration=252.261420321 podStartE2EDuration="4m14.143020937s" totalImagesPullingTime="1.881600616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:37:34.141944891 +0000 UTC m=+289.656680194" watchObservedRunningTime="2026-07-10 03:37:34.143020937 +0000 UTC m=+289.657756239"
Jul 10 03:37:35.222074 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:35.222039    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:35.222522 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:35.222097    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:35.227051 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:35.227028    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:36.132972 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:36.132945    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:37:44.128066 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:44.128034    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wszvx"
Jul 10 03:37:44.914542 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:44.914516    2571 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 03:37:45.545275 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:45.545237    2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp" podUID="027627e2-0a96-4507-a3be-03e811133cd6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 10 03:37:55.545546 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:37:55.545504    2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp" podUID="027627e2-0a96-4507-a3be-03e811133cd6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 10 03:38:05.544832 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:38:05.544788    2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp" podUID="027627e2-0a96-4507-a3be-03e811133cd6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 10 03:38:05.545273 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:38:05.544877    2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp"
Jul 10 03:38:05.545435 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:38:05.545403    2571 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"c2830cfa48a9608100cf36657289eae496ec6c53ac157b6052aa844fed86c660"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 10 03:38:05.545491 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:38:05.545473    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp" podUID="027627e2-0a96-4507-a3be-03e811133cd6" containerName="service-proxy" containerID="cri-o://c2830cfa48a9608100cf36657289eae496ec6c53ac157b6052aa844fed86c660" gracePeriod=30
Jul 10 03:38:06.229582 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:38:06.229547    2571 generic.go:356] "Generic (PLEG): container finished" podID="027627e2-0a96-4507-a3be-03e811133cd6" containerID="c2830cfa48a9608100cf36657289eae496ec6c53ac157b6052aa844fed86c660" exitCode=2
Jul 10 03:38:06.229755 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:38:06.229614    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp" event={"ID":"027627e2-0a96-4507-a3be-03e811133cd6","Type":"ContainerDied","Data":"c2830cfa48a9608100cf36657289eae496ec6c53ac157b6052aa844fed86c660"}
Jul 10 03:38:06.229755 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:38:06.229659    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-85bff74d6-zvhrp" event={"ID":"027627e2-0a96-4507-a3be-03e811133cd6","Type":"ContainerStarted","Data":"d1ad9a2f12b37da4d76ed85745e3aebc7915dd25d998880ab2fac672472d5006"}
Jul 10 03:38:43.421269 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:38:43.421189    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dc85db69d-xqt85"]
Jul 10 03:39:08.440701 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.440640    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5dc85db69d-xqt85" podUID="c4ab6a03-8755-42aa-8b22-4218e473873e" containerName="console" containerID="cri-o://2f2d0d6275548114f9cba5071a54355b037886ec6a52e0a66543d584faa291a6" gracePeriod=15
Jul 10 03:39:08.691065 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.691004    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:39:08.807024 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.806989    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwq4z\" (UniqueName: \"kubernetes.io/projected/c4ab6a03-8755-42aa-8b22-4218e473873e-kube-api-access-dwq4z\") pod \"c4ab6a03-8755-42aa-8b22-4218e473873e\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") "
Jul 10 03:39:08.807024 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.807027    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4ab6a03-8755-42aa-8b22-4218e473873e-console-oauth-config\") pod \"c4ab6a03-8755-42aa-8b22-4218e473873e\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") "
Jul 10 03:39:08.807286 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.807052    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4ab6a03-8755-42aa-8b22-4218e473873e-console-serving-cert\") pod \"c4ab6a03-8755-42aa-8b22-4218e473873e\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") "
Jul 10 03:39:08.807286 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.807070    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-service-ca\") pod \"c4ab6a03-8755-42aa-8b22-4218e473873e\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") "
Jul 10 03:39:08.807286 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.807087    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-console-config\") pod \"c4ab6a03-8755-42aa-8b22-4218e473873e\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") "
Jul 10 03:39:08.807286 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.807221    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-oauth-serving-cert\") pod \"c4ab6a03-8755-42aa-8b22-4218e473873e\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") "
Jul 10 03:39:08.807527 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.807281    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-trusted-ca-bundle\") pod \"c4ab6a03-8755-42aa-8b22-4218e473873e\" (UID: \"c4ab6a03-8755-42aa-8b22-4218e473873e\") "
Jul 10 03:39:08.807586 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.807531    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-console-config" (OuterVolumeSpecName: "console-config") pod "c4ab6a03-8755-42aa-8b22-4218e473873e" (UID: "c4ab6a03-8755-42aa-8b22-4218e473873e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:39:08.807586 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.807559    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-service-ca" (OuterVolumeSpecName: "service-ca") pod "c4ab6a03-8755-42aa-8b22-4218e473873e" (UID: "c4ab6a03-8755-42aa-8b22-4218e473873e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:39:08.807702 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.807634    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c4ab6a03-8755-42aa-8b22-4218e473873e" (UID: "c4ab6a03-8755-42aa-8b22-4218e473873e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:39:08.807900 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.807873    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c4ab6a03-8755-42aa-8b22-4218e473873e" (UID: "c4ab6a03-8755-42aa-8b22-4218e473873e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:39:08.809352 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.809330    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4ab6a03-8755-42aa-8b22-4218e473873e-kube-api-access-dwq4z" (OuterVolumeSpecName: "kube-api-access-dwq4z") pod "c4ab6a03-8755-42aa-8b22-4218e473873e" (UID: "c4ab6a03-8755-42aa-8b22-4218e473873e"). InnerVolumeSpecName "kube-api-access-dwq4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:39:08.809352 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.809340    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4ab6a03-8755-42aa-8b22-4218e473873e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c4ab6a03-8755-42aa-8b22-4218e473873e" (UID: "c4ab6a03-8755-42aa-8b22-4218e473873e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:39:08.809515 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.809353    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4ab6a03-8755-42aa-8b22-4218e473873e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c4ab6a03-8755-42aa-8b22-4218e473873e" (UID: "c4ab6a03-8755-42aa-8b22-4218e473873e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:39:08.908186 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.908140    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-trusted-ca-bundle\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:39:08.908186 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.908177    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwq4z\" (UniqueName: \"kubernetes.io/projected/c4ab6a03-8755-42aa-8b22-4218e473873e-kube-api-access-dwq4z\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:39:08.908186 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.908190    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4ab6a03-8755-42aa-8b22-4218e473873e-console-oauth-config\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:39:08.908416 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.908206    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4ab6a03-8755-42aa-8b22-4218e473873e-console-serving-cert\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:39:08.908416 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.908218    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-service-ca\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:39:08.908416 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.908229    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-console-config\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:39:08.908416 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:08.908238    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4ab6a03-8755-42aa-8b22-4218e473873e-oauth-serving-cert\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:39:09.443805 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:09.443776    2571 generic.go:356] "Generic (PLEG): container finished" podID="c4ab6a03-8755-42aa-8b22-4218e473873e" containerID="2f2d0d6275548114f9cba5071a54355b037886ec6a52e0a66543d584faa291a6" exitCode=0
Jul 10 03:39:09.444237 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:09.443817    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc85db69d-xqt85"
Jul 10 03:39:09.444237 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:09.443824    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc85db69d-xqt85" event={"ID":"c4ab6a03-8755-42aa-8b22-4218e473873e","Type":"ContainerDied","Data":"2f2d0d6275548114f9cba5071a54355b037886ec6a52e0a66543d584faa291a6"}
Jul 10 03:39:09.444237 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:09.443867    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc85db69d-xqt85" event={"ID":"c4ab6a03-8755-42aa-8b22-4218e473873e","Type":"ContainerDied","Data":"536f9789cd04e87807b7401008a5bd0de02acb0b76aba14e7e5cf7348f55c361"}
Jul 10 03:39:09.444237 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:09.443889    2571 scope.go:117] "RemoveContainer" containerID="2f2d0d6275548114f9cba5071a54355b037886ec6a52e0a66543d584faa291a6"
Jul 10 03:39:09.451912 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:09.451892    2571 scope.go:117] "RemoveContainer" containerID="2f2d0d6275548114f9cba5071a54355b037886ec6a52e0a66543d584faa291a6"
Jul 10 03:39:09.452203 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:39:09.452179    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f2d0d6275548114f9cba5071a54355b037886ec6a52e0a66543d584faa291a6\": container with ID starting with 2f2d0d6275548114f9cba5071a54355b037886ec6a52e0a66543d584faa291a6 not found: ID does not exist" containerID="2f2d0d6275548114f9cba5071a54355b037886ec6a52e0a66543d584faa291a6"
Jul 10 03:39:09.452273 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:09.452212    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f2d0d6275548114f9cba5071a54355b037886ec6a52e0a66543d584faa291a6"} err="failed to get container status \"2f2d0d6275548114f9cba5071a54355b037886ec6a52e0a66543d584faa291a6\": rpc error: code = NotFound desc = could not find container \"2f2d0d6275548114f9cba5071a54355b037886ec6a52e0a66543d584faa291a6\": container with ID starting with 2f2d0d6275548114f9cba5071a54355b037886ec6a52e0a66543d584faa291a6 not found: ID does not exist"
Jul 10 03:39:09.458040 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:09.458015    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dc85db69d-xqt85"]
Jul 10 03:39:09.461098 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:09.461073    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5dc85db69d-xqt85"]
Jul 10 03:39:11.124637 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:39:11.124605    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4ab6a03-8755-42aa-8b22-4218e473873e" path="/var/lib/kubelet/pods/c4ab6a03-8755-42aa-8b22-4218e473873e/volumes"
Jul 10 03:40:21.100007 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.099930    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"]
Jul 10 03:40:21.100452 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.100164    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ab6a03-8755-42aa-8b22-4218e473873e" containerName="console"
Jul 10 03:40:21.100452 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.100174    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ab6a03-8755-42aa-8b22-4218e473873e" containerName="console"
Jul 10 03:40:21.100452 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.100217    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ab6a03-8755-42aa-8b22-4218e473873e" containerName="console"
Jul 10 03:40:21.105643 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.105608    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.108593 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.108572    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 03:40:21.108593 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.108581    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 03:40:21.108792 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.108569    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 03:40:21.108792 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.108679    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 03:40:21.108792 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.108726    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mzf2m\""
Jul 10 03:40:21.117999 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.117975    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"]
Jul 10 03:40:21.225986 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.225953    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0c25b42d-cf89-4ee2-945a-44c91292719e-manifests\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.226190 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.225991    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0c25b42d-cf89-4ee2-945a-44c91292719e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.226190 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.226021    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c25b42d-cf89-4ee2-945a-44c91292719e-tmp\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.226190 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.226125    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4zlj\" (UniqueName: \"kubernetes.io/projected/0c25b42d-cf89-4ee2-945a-44c91292719e-kube-api-access-s4zlj\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.226190 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.226167    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c25b42d-cf89-4ee2-945a-44c91292719e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.327594 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.327560    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0c25b42d-cf89-4ee2-945a-44c91292719e-manifests\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.327594 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.327595    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0c25b42d-cf89-4ee2-945a-44c91292719e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.327838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.327612    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c25b42d-cf89-4ee2-945a-44c91292719e-tmp\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.327838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.327642    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4zlj\" (UniqueName: \"kubernetes.io/projected/0c25b42d-cf89-4ee2-945a-44c91292719e-kube-api-access-s4zlj\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.327838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.327661    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c25b42d-cf89-4ee2-945a-44c91292719e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.328033 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.328013    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0c25b42d-cf89-4ee2-945a-44c91292719e-manifests\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.329970 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.329939    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c25b42d-cf89-4ee2-945a-44c91292719e-tmp\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.330064 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.330000    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0c25b42d-cf89-4ee2-945a-44c91292719e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.330103 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.330093    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c25b42d-cf89-4ee2-945a-44c91292719e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.334447 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.334425    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4zlj\" (UniqueName: \"kubernetes.io/projected/0c25b42d-cf89-4ee2-945a-44c91292719e-kube-api-access-s4zlj\") pod \"opendatahub-operator-controller-manager-7d676bbf4b-j4p7p\" (UID: \"0c25b42d-cf89-4ee2-945a-44c91292719e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.415708 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.415612    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:21.564816 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.564722    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"]
Jul 10 03:40:21.567694 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:40:21.567667    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c25b42d_cf89_4ee2_945a_44c91292719e.slice/crio-41689521b3c71dba87c95b4842e07fdb306d25ab2adf5bed947c9aea8f3264a6 WatchSource:0}: Error finding container 41689521b3c71dba87c95b4842e07fdb306d25ab2adf5bed947c9aea8f3264a6: Status 404 returned error can't find the container with id 41689521b3c71dba87c95b4842e07fdb306d25ab2adf5bed947c9aea8f3264a6
Jul 10 03:40:21.569343 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.569323    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 03:40:21.682113 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:21.682077    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p" event={"ID":"0c25b42d-cf89-4ee2-945a-44c91292719e","Type":"ContainerStarted","Data":"41689521b3c71dba87c95b4842e07fdb306d25ab2adf5bed947c9aea8f3264a6"}
Jul 10 03:40:24.695566 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:24.695513    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p" event={"ID":"0c25b42d-cf89-4ee2-945a-44c91292719e","Type":"ContainerStarted","Data":"01f26d9f2e122b829400ce2a01f5681bcbe91d9452cb98be3cd5096fb046f676"}
Jul 10 03:40:25.700943 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:25.700907    2571 generic.go:356] "Generic (PLEG): container finished" podID="0c25b42d-cf89-4ee2-945a-44c91292719e" containerID="01f26d9f2e122b829400ce2a01f5681bcbe91d9452cb98be3cd5096fb046f676" exitCode=0
Jul 10 03:40:25.701303 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:25.700966    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p" event={"ID":"0c25b42d-cf89-4ee2-945a-44c91292719e","Type":"ContainerDied","Data":"01f26d9f2e122b829400ce2a01f5681bcbe91d9452cb98be3cd5096fb046f676"}
Jul 10 03:40:26.706050 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:26.706007    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p" event={"ID":"0c25b42d-cf89-4ee2-945a-44c91292719e","Type":"ContainerStarted","Data":"e7c2772d4303d7f62762502390f2c491cb4facb876c5fb997fd93c27bd62df6b"}
Jul 10 03:40:26.706655 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:26.706102    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:26.726247 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:26.726187    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p" podStartSLOduration=1.393465757 podStartE2EDuration="5.726165941s" totalImagesPullingTime="3.282659581s" totalInitContainerRuntime="1.050040603s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:40:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:40:26.724442132 +0000 UTC m=+462.239177444" watchObservedRunningTime="2026-07-10 03:40:26.726165941 +0000 UTC m=+462.240901244"
Jul 10 03:40:37.711190 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:37.711163    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d676bbf4b-j4p7p"
Jul 10 03:40:40.017417 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.017379    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mg5lt"]
Jul 10 03:40:40.020704 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.020681    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt"
Jul 10 03:40:40.022999 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.022977    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 10 03:40:40.023126 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.023106    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6tjdk\""
Jul 10 03:40:40.031133 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.031107    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mg5lt"]
Jul 10 03:40:40.181682 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.181647    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbr7d\" (UniqueName: \"kubernetes.io/projected/6052a68a-6878-4bec-ab1d-27c9acc6f346-kube-api-access-kbr7d\") pod \"odh-model-controller-7998c67dc4-mg5lt\" (UID: \"6052a68a-6878-4bec-ab1d-27c9acc6f346\") " pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt"
Jul 10 03:40:40.181682 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.181693    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6052a68a-6878-4bec-ab1d-27c9acc6f346-cert\") pod \"odh-model-controller-7998c67dc4-mg5lt\" (UID: \"6052a68a-6878-4bec-ab1d-27c9acc6f346\") " pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt"
Jul 10 03:40:40.282693 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.282588    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbr7d\" (UniqueName: \"kubernetes.io/projected/6052a68a-6878-4bec-ab1d-27c9acc6f346-kube-api-access-kbr7d\") pod \"odh-model-controller-7998c67dc4-mg5lt\" (UID: \"6052a68a-6878-4bec-ab1d-27c9acc6f346\") " pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt"
Jul 10 03:40:40.282693 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.282639    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6052a68a-6878-4bec-ab1d-27c9acc6f346-cert\") pod \"odh-model-controller-7998c67dc4-mg5lt\" (UID: \"6052a68a-6878-4bec-ab1d-27c9acc6f346\") " pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt"
Jul 10 03:40:40.282879 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:40:40.282796    2571 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 10 03:40:40.282914 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:40:40.282882    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6052a68a-6878-4bec-ab1d-27c9acc6f346-cert podName:6052a68a-6878-4bec-ab1d-27c9acc6f346 nodeName:}" failed. No retries permitted until 2026-07-10 03:40:40.782866151 +0000 UTC m=+476.297601432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6052a68a-6878-4bec-ab1d-27c9acc6f346-cert") pod "odh-model-controller-7998c67dc4-mg5lt" (UID: "6052a68a-6878-4bec-ab1d-27c9acc6f346") : secret "odh-model-controller-webhook-cert" not found
Jul 10 03:40:40.290941 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.290910    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbr7d\" (UniqueName: \"kubernetes.io/projected/6052a68a-6878-4bec-ab1d-27c9acc6f346-kube-api-access-kbr7d\") pod \"odh-model-controller-7998c67dc4-mg5lt\" (UID: \"6052a68a-6878-4bec-ab1d-27c9acc6f346\") " pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt"
Jul 10 03:40:40.625564 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.625484    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-sxbh5"]
Jul 10 03:40:40.628140 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.628125    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5"
Jul 10 03:40:40.630175 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.630150    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 10 03:40:40.640480 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.640440    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-sxbh5"]
Jul 10 03:40:40.787830 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.787799    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6052a68a-6878-4bec-ab1d-27c9acc6f346-cert\") pod \"odh-model-controller-7998c67dc4-mg5lt\" (UID: \"6052a68a-6878-4bec-ab1d-27c9acc6f346\") " pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt"
Jul 10 03:40:40.788016 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.787846    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb4n7\" (UniqueName: \"kubernetes.io/projected/d549e6f8-9589-4a94-a8e3-49d44eb3276c-kube-api-access-gb4n7\") pod \"model-serving-api-54d89bd5fc-sxbh5\" (UID: \"d549e6f8-9589-4a94-a8e3-49d44eb3276c\") " pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5"
Jul 10 03:40:40.788016 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.787892    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d549e6f8-9589-4a94-a8e3-49d44eb3276c-tls-certs\") pod \"model-serving-api-54d89bd5fc-sxbh5\" (UID: \"d549e6f8-9589-4a94-a8e3-49d44eb3276c\") " pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5"
Jul 10 03:40:40.790156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.790125    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6052a68a-6878-4bec-ab1d-27c9acc6f346-cert\") pod \"odh-model-controller-7998c67dc4-mg5lt\" (UID: \"6052a68a-6878-4bec-ab1d-27c9acc6f346\") " pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt"
Jul 10 03:40:40.889184 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.889097    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb4n7\" (UniqueName: \"kubernetes.io/projected/d549e6f8-9589-4a94-a8e3-49d44eb3276c-kube-api-access-gb4n7\") pod \"model-serving-api-54d89bd5fc-sxbh5\" (UID: \"d549e6f8-9589-4a94-a8e3-49d44eb3276c\") " pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5"
Jul 10 03:40:40.889184 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.889143    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d549e6f8-9589-4a94-a8e3-49d44eb3276c-tls-certs\") pod \"model-serving-api-54d89bd5fc-sxbh5\" (UID: \"d549e6f8-9589-4a94-a8e3-49d44eb3276c\") " pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5"
Jul 10 03:40:40.889361 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:40:40.889258    2571 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 10 03:40:40.889361 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:40:40.889314    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d549e6f8-9589-4a94-a8e3-49d44eb3276c-tls-certs podName:d549e6f8-9589-4a94-a8e3-49d44eb3276c nodeName:}" failed. No retries permitted until 2026-07-10 03:40:41.389297908 +0000 UTC m=+476.904033188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d549e6f8-9589-4a94-a8e3-49d44eb3276c-tls-certs") pod "model-serving-api-54d89bd5fc-sxbh5" (UID: "d549e6f8-9589-4a94-a8e3-49d44eb3276c") : secret "model-serving-api-tls" not found
Jul 10 03:40:40.897481 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.897422    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb4n7\" (UniqueName: \"kubernetes.io/projected/d549e6f8-9589-4a94-a8e3-49d44eb3276c-kube-api-access-gb4n7\") pod \"model-serving-api-54d89bd5fc-sxbh5\" (UID: \"d549e6f8-9589-4a94-a8e3-49d44eb3276c\") " pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5"
Jul 10 03:40:40.932387 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:40.932347    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt"
Jul 10 03:40:41.064816 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.064791    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mg5lt"]
Jul 10 03:40:41.067760 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:40:41.067732    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6052a68a_6878_4bec_ab1d_27c9acc6f346.slice/crio-068800c17c4eb329ea138caffc169d17efa02ab21a3451aa5f29f9443d99d8bf WatchSource:0}: Error finding container 068800c17c4eb329ea138caffc169d17efa02ab21a3451aa5f29f9443d99d8bf: Status 404 returned error can't find the container with id 068800c17c4eb329ea138caffc169d17efa02ab21a3451aa5f29f9443d99d8bf
Jul 10 03:40:41.393551 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.393508    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d549e6f8-9589-4a94-a8e3-49d44eb3276c-tls-certs\") pod \"model-serving-api-54d89bd5fc-sxbh5\" (UID: \"d549e6f8-9589-4a94-a8e3-49d44eb3276c\") " pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5"
Jul 10 03:40:41.396168 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.396140    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d549e6f8-9589-4a94-a8e3-49d44eb3276c-tls-certs\") pod \"model-serving-api-54d89bd5fc-sxbh5\" (UID: \"d549e6f8-9589-4a94-a8e3-49d44eb3276c\") " pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5"
Jul 10 03:40:41.532760 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.532722    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"]
Jul 10 03:40:41.537258 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.537233    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"
Jul 10 03:40:41.538481 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.538429    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5"
Jul 10 03:40:41.539486 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.539433    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 03:40:41.539621 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.539485    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 10 03:40:41.540146 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.540128    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 10 03:40:41.540146 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.540136    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2746x\""
Jul 10 03:40:41.540270 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.540183    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 03:40:41.544812 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.544790    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"]
Jul 10 03:40:41.678781 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.678750    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-sxbh5"]
Jul 10 03:40:41.682022 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:40:41.681995    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd549e6f8_9589_4a94_a8e3_49d44eb3276c.slice/crio-72ac4d8df6bd2101a11e533733acebf15ed9ac5b0d57e154f9bc873ac2b9db84 WatchSource:0}: Error finding container 72ac4d8df6bd2101a11e533733acebf15ed9ac5b0d57e154f9bc873ac2b9db84: Status 404 returned error can't find the container with id 72ac4d8df6bd2101a11e533733acebf15ed9ac5b0d57e154f9bc873ac2b9db84
Jul 10 03:40:41.696097 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.696073    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a-tls-certs\") pod \"kube-auth-proxy-d9645bfdd-2grjn\" (UID: \"4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"
Jul 10 03:40:41.696234 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.696112    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjgrx\" (UniqueName: \"kubernetes.io/projected/4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a-kube-api-access-hjgrx\") pod \"kube-auth-proxy-d9645bfdd-2grjn\" (UID: \"4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"
Jul 10 03:40:41.696303 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.696251    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a-tmp\") pod \"kube-auth-proxy-d9645bfdd-2grjn\" (UID: \"4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"
Jul 10 03:40:41.760319 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.760282    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5" event={"ID":"d549e6f8-9589-4a94-a8e3-49d44eb3276c","Type":"ContainerStarted","Data":"72ac4d8df6bd2101a11e533733acebf15ed9ac5b0d57e154f9bc873ac2b9db84"}
Jul 10 03:40:41.761604 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.761572    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt" event={"ID":"6052a68a-6878-4bec-ab1d-27c9acc6f346","Type":"ContainerStarted","Data":"068800c17c4eb329ea138caffc169d17efa02ab21a3451aa5f29f9443d99d8bf"}
Jul 10 03:40:41.797075 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.797035    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a-tmp\") pod \"kube-auth-proxy-d9645bfdd-2grjn\" (UID: \"4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"
Jul 10 03:40:41.797247 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.797088    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a-tls-certs\") pod \"kube-auth-proxy-d9645bfdd-2grjn\" (UID: \"4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"
Jul 10 03:40:41.797247 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.797126    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjgrx\" (UniqueName: \"kubernetes.io/projected/4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a-kube-api-access-hjgrx\") pod \"kube-auth-proxy-d9645bfdd-2grjn\" (UID: \"4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"
Jul 10 03:40:41.797350 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:40:41.797329    2571 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 10 03:40:41.797436 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:40:41.797407    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a-tls-certs podName:4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a nodeName:}" failed. No retries permitted until 2026-07-10 03:40:42.297386137 +0000 UTC m=+477.812121423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a-tls-certs") pod "kube-auth-proxy-d9645bfdd-2grjn" (UID: "4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a") : secret "kube-auth-proxy-tls" not found
Jul 10 03:40:41.799435 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.799404    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a-tmp\") pod \"kube-auth-proxy-d9645bfdd-2grjn\" (UID: \"4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"
Jul 10 03:40:41.808540 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:41.808513    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjgrx\" (UniqueName: \"kubernetes.io/projected/4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a-kube-api-access-hjgrx\") pod \"kube-auth-proxy-d9645bfdd-2grjn\" (UID: \"4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"
Jul 10 03:40:42.302828 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.302791    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a-tls-certs\") pod \"kube-auth-proxy-d9645bfdd-2grjn\" (UID: \"4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"
Jul 10 03:40:42.306546 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.306491    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a-tls-certs\") pod \"kube-auth-proxy-d9645bfdd-2grjn\" (UID: \"4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a\") " pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"
Jul 10 03:40:42.466705 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.466488    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"
Jul 10 03:40:42.472149 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.471500    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"]
Jul 10 03:40:42.474830 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.474750    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.477173 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.477151    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 10 03:40:42.477510 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.477394    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 03:40:42.477621 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.477560    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qdvfk\""
Jul 10 03:40:42.477819 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.477740    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 10 03:40:42.478224 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.478072    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 10 03:40:42.490211 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.490113    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"]
Jul 10 03:40:42.606961 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.606056    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.606961 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.606134    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvcnj\" (UniqueName: \"kubernetes.io/projected/794f1d63-0711-425b-a327-32d4944ce547-kube-api-access-vvcnj\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.606961 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.606173    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-cacerts\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.606961 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.606201    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/794f1d63-0711-425b-a327-32d4944ce547-local-certs\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.606961 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.606233    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/794f1d63-0711-425b-a327-32d4944ce547-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.606961 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.606372    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/794f1d63-0711-425b-a327-32d4944ce547-istio-token\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.606961 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.606437    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.647257 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.647138    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn"]
Jul 10 03:40:42.708040 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.707493    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/794f1d63-0711-425b-a327-32d4944ce547-istio-token\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.708040 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.707554    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.708040 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.707591    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.708040 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.707716    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvcnj\" (UniqueName: \"kubernetes.io/projected/794f1d63-0711-425b-a327-32d4944ce547-kube-api-access-vvcnj\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.708040 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.707760    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-cacerts\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.708040 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.707792    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/794f1d63-0711-425b-a327-32d4944ce547-local-certs\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.708040 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.707828    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/794f1d63-0711-425b-a327-32d4944ce547-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.708534 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.708412    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/794f1d63-0711-425b-a327-32d4944ce547-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.711347 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.711323    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.712743 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.712026    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.712743 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.712342    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/794f1d63-0711-425b-a327-32d4944ce547-local-certs\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.713137 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.713114    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-cacerts\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.716037 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.715984    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvcnj\" (UniqueName: \"kubernetes.io/projected/794f1d63-0711-425b-a327-32d4944ce547-kube-api-access-vvcnj\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.716176 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.716149    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/794f1d63-0711-425b-a327-32d4944ce547-istio-token\") pod \"istiod-openshift-gateway-7574459656-jfljw\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:42.806913 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:42.806879    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:43.945083 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:40:43.945052    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bcc8dc3_bac1_49c8_b038_d2bebfe5df3a.slice/crio-99212895fd90278cbb9f51dd88b0e78cc5b0bd460f91e0713e917a91454a1e5f WatchSource:0}: Error finding container 99212895fd90278cbb9f51dd88b0e78cc5b0bd460f91e0713e917a91454a1e5f: Status 404 returned error can't find the container with id 99212895fd90278cbb9f51dd88b0e78cc5b0bd460f91e0713e917a91454a1e5f
Jul 10 03:40:44.500157 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:44.500127    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"]
Jul 10 03:40:44.503261 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:40:44.503222    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794f1d63_0711_425b_a327_32d4944ce547.slice/crio-a6b55b8ff59ca8db6158d0498a33a9857833c42d2a16cb31fcd5ace03084e739 WatchSource:0}: Error finding container a6b55b8ff59ca8db6158d0498a33a9857833c42d2a16cb31fcd5ace03084e739: Status 404 returned error can't find the container with id a6b55b8ff59ca8db6158d0498a33a9857833c42d2a16cb31fcd5ace03084e739
Jul 10 03:40:44.777894 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:44.777853    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw" event={"ID":"794f1d63-0711-425b-a327-32d4944ce547","Type":"ContainerStarted","Data":"a6b55b8ff59ca8db6158d0498a33a9857833c42d2a16cb31fcd5ace03084e739"}
Jul 10 03:40:44.779716 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:44.779672    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt" event={"ID":"6052a68a-6878-4bec-ab1d-27c9acc6f346","Type":"ContainerStarted","Data":"869c44c956077f3f4d57baf5b4541ed6742f6593d48b5b0d61e8356ccb1b4f6e"}
Jul 10 03:40:44.779850 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:44.779795    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt"
Jul 10 03:40:44.781031 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:44.781005    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn" event={"ID":"4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a","Type":"ContainerStarted","Data":"99212895fd90278cbb9f51dd88b0e78cc5b0bd460f91e0713e917a91454a1e5f"}
Jul 10 03:40:44.782965 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:44.782944    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5" event={"ID":"d549e6f8-9589-4a94-a8e3-49d44eb3276c","Type":"ContainerStarted","Data":"7c40e770fc39db391cb699438591a317cf1a86fd93d99978c550944129e5e987"}
Jul 10 03:40:44.783096 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:44.783072    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5"
Jul 10 03:40:44.797060 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:44.797010    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt" podStartSLOduration=2.511789355 podStartE2EDuration="5.796993447s" totalImagesPullingTime="3.285204092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:40:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:40:44.79518545 +0000 UTC m=+480.309920754" watchObservedRunningTime="2026-07-10 03:40:44.796993447 +0000 UTC m=+480.311728749"
Jul 10 03:40:44.811489 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:44.811418    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5" podStartSLOduration=2.143377386 podStartE2EDuration="4.811401743s" totalImagesPullingTime="2.668024357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:40:44.81112572 +0000 UTC m=+480.325861039" watchObservedRunningTime="2026-07-10 03:40:44.811401743 +0000 UTC m=+480.326137043"
Jul 10 03:40:46.452317 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:46.452292    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 10 03:40:46.794602 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:46.794498    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn" event={"ID":"4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a","Type":"ContainerStarted","Data":"f105f93a98526c4c396783b10e02c51e3877b2c756786d3a5db3744396090b54"}
Jul 10 03:40:46.810112 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:46.810052    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-d9645bfdd-2grjn" podStartSLOduration=3.308157648 podStartE2EDuration="5.810033901s" totalImagesPullingTime="2.501876253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:40:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:40:46.809375035 +0000 UTC m=+482.324110349" watchObservedRunningTime="2026-07-10 03:40:46.810033901 +0000 UTC m=+482.324769203"
Jul 10 03:40:47.455010 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:47.454965    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 03:40:47.455268 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:47.455042    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 03:40:47.799849 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:47.799748    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw" event={"ID":"794f1d63-0711-425b-a327-32d4944ce547","Type":"ContainerStarted","Data":"a8c6dfd628ea3076f3692c3e552c24f6d50be7f8251f226d63df7b98cf97f5a3"}
Jul 10 03:40:48.804425 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:48.804381    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:48.806167 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:48.806124    2571 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-jfljw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 03:40:48.806353 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:48.806174    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw" podUID="794f1d63-0711-425b-a327-32d4944ce547" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 03:40:49.517766 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.517719    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw" podStartSLOduration=4.568711906 podStartE2EDuration="7.51770124s" totalImagesPullingTime="2.948989334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:40:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:40:47.82697014 +0000 UTC m=+483.341705442" watchObservedRunningTime="2026-07-10 03:40:49.51770124 +0000 UTC m=+485.032436541"
Jul 10 03:40:49.518356 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.518322    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"]
Jul 10 03:40:49.521541 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.521527    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.525209 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.525188    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qgc6c\""
Jul 10 03:40:49.536802 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.536783    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"]
Jul 10 03:40:49.564337 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.564310    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.564496 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.564345    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.564496 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.564365    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e0815c6-dc6d-4a74-8621-fbe550124c15-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.564496 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.564404    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e0815c6-dc6d-4a74-8621-fbe550124c15-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.564496 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.564481    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.564616 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.564512    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.564616 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.564535    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.564616 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.564560    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjz9r\" (UniqueName: \"kubernetes.io/projected/6e0815c6-dc6d-4a74-8621-fbe550124c15-kube-api-access-pjz9r\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.564702 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.564609    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e0815c6-dc6d-4a74-8621-fbe550124c15-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.665312 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.665275    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e0815c6-dc6d-4a74-8621-fbe550124c15-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.665528 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.665332    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.665528 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.665354    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.665528 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.665374    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e0815c6-dc6d-4a74-8621-fbe550124c15-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.665528 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.665388    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e0815c6-dc6d-4a74-8621-fbe550124c15-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.665528 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.665418    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.665528 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.665448    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.665528 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.665506    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.665875 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.665535    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjz9r\" (UniqueName: \"kubernetes.io/projected/6e0815c6-dc6d-4a74-8621-fbe550124c15-kube-api-access-pjz9r\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.665925 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.665886    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.665977 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.665939    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.666058 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.666036    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.666150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.666127    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.666317 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.666289    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e0815c6-dc6d-4a74-8621-fbe550124c15-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.667826 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.667797    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e0815c6-dc6d-4a74-8621-fbe550124c15-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.667922 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.667843    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e0815c6-dc6d-4a74-8621-fbe550124c15-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.684079 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.684042    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e0815c6-dc6d-4a74-8621-fbe550124c15-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.684688 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.684669    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjz9r\" (UniqueName: \"kubernetes.io/projected/6e0815c6-dc6d-4a74-8621-fbe550124c15-kube-api-access-pjz9r\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4599pm\" (UID: \"6e0815c6-dc6d-4a74-8621-fbe550124c15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.808931 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.808850    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:49.831640 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.831614    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:49.992183 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:49.992162    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"]
Jul 10 03:40:49.994597 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:40:49.994568    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e0815c6_dc6d_4a74_8621_fbe550124c15.slice/crio-0d02eed22860889e953acf869a0cc60b97bb3d6eface23495c7b5154a5dc3cb1 WatchSource:0}: Error finding container 0d02eed22860889e953acf869a0cc60b97bb3d6eface23495c7b5154a5dc3cb1: Status 404 returned error can't find the container with id 0d02eed22860889e953acf869a0cc60b97bb3d6eface23495c7b5154a5dc3cb1
Jul 10 03:40:50.813526 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:50.813490    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm" event={"ID":"6e0815c6-dc6d-4a74-8621-fbe550124c15","Type":"ContainerStarted","Data":"0d02eed22860889e953acf869a0cc60b97bb3d6eface23495c7b5154a5dc3cb1"}
Jul 10 03:40:51.274083 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.274051    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"]
Jul 10 03:40:51.278676 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.278658    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"
Jul 10 03:40:51.281159 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.281136    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jjl2z\""
Jul 10 03:40:51.281649 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.281631    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg"]
Jul 10 03:40:51.282205 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.282190    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 10 03:40:51.284673 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.284655    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg"
Jul 10 03:40:51.287270 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.287029    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 10 03:40:51.287270 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.287267    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-pjkdg\""
Jul 10 03:40:51.289124 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.289102    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"]
Jul 10 03:40:51.296479 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.296446    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg"]
Jul 10 03:40:51.378569 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.378532    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/008f3b93-2fa3-4193-9eb5-c49b118edf92-cert\") pod \"kserve-controller-manager-d6f4b65b8-g4zvw\" (UID: \"008f3b93-2fa3-4193-9eb5-c49b118edf92\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"
Jul 10 03:40:51.378757 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.378628    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tskqg\" (UniqueName: \"kubernetes.io/projected/008f3b93-2fa3-4193-9eb5-c49b118edf92-kube-api-access-tskqg\") pod \"kserve-controller-manager-d6f4b65b8-g4zvw\" (UID: \"008f3b93-2fa3-4193-9eb5-c49b118edf92\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"
Jul 10 03:40:51.479024 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.478995    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51022770-1212-4d2f-abed-3853fc50624c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9qwfg\" (UID: \"51022770-1212-4d2f-abed-3853fc50624c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg"
Jul 10 03:40:51.479188 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.479032    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/008f3b93-2fa3-4193-9eb5-c49b118edf92-cert\") pod \"kserve-controller-manager-d6f4b65b8-g4zvw\" (UID: \"008f3b93-2fa3-4193-9eb5-c49b118edf92\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"
Jul 10 03:40:51.479188 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.479067    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tskqg\" (UniqueName: \"kubernetes.io/projected/008f3b93-2fa3-4193-9eb5-c49b118edf92-kube-api-access-tskqg\") pod \"kserve-controller-manager-d6f4b65b8-g4zvw\" (UID: \"008f3b93-2fa3-4193-9eb5-c49b118edf92\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"
Jul 10 03:40:51.479188 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.479084    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsdx7\" (UniqueName: \"kubernetes.io/projected/51022770-1212-4d2f-abed-3853fc50624c-kube-api-access-rsdx7\") pod \"llmisvc-controller-manager-777c6bd7d5-9qwfg\" (UID: \"51022770-1212-4d2f-abed-3853fc50624c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg"
Jul 10 03:40:51.479188 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:40:51.479162    2571 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 10 03:40:51.479352 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:40:51.479226    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/008f3b93-2fa3-4193-9eb5-c49b118edf92-cert podName:008f3b93-2fa3-4193-9eb5-c49b118edf92 nodeName:}" failed. No retries permitted until 2026-07-10 03:40:51.979207929 +0000 UTC m=+487.493943210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/008f3b93-2fa3-4193-9eb5-c49b118edf92-cert") pod "kserve-controller-manager-d6f4b65b8-g4zvw" (UID: "008f3b93-2fa3-4193-9eb5-c49b118edf92") : secret "kserve-webhook-server-cert" not found
Jul 10 03:40:51.492923 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.492899    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tskqg\" (UniqueName: \"kubernetes.io/projected/008f3b93-2fa3-4193-9eb5-c49b118edf92-kube-api-access-tskqg\") pod \"kserve-controller-manager-d6f4b65b8-g4zvw\" (UID: \"008f3b93-2fa3-4193-9eb5-c49b118edf92\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"
Jul 10 03:40:51.579754 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.579655    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsdx7\" (UniqueName: \"kubernetes.io/projected/51022770-1212-4d2f-abed-3853fc50624c-kube-api-access-rsdx7\") pod \"llmisvc-controller-manager-777c6bd7d5-9qwfg\" (UID: \"51022770-1212-4d2f-abed-3853fc50624c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg"
Jul 10 03:40:51.579905 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.579758    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51022770-1212-4d2f-abed-3853fc50624c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9qwfg\" (UID: \"51022770-1212-4d2f-abed-3853fc50624c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg"
Jul 10 03:40:51.582344 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.582297    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51022770-1212-4d2f-abed-3853fc50624c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9qwfg\" (UID: \"51022770-1212-4d2f-abed-3853fc50624c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg"
Jul 10 03:40:51.599912 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.599884    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsdx7\" (UniqueName: \"kubernetes.io/projected/51022770-1212-4d2f-abed-3853fc50624c-kube-api-access-rsdx7\") pod \"llmisvc-controller-manager-777c6bd7d5-9qwfg\" (UID: \"51022770-1212-4d2f-abed-3853fc50624c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg"
Jul 10 03:40:51.898213 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.898123    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg"
Jul 10 03:40:51.983740 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:51.983710    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/008f3b93-2fa3-4193-9eb5-c49b118edf92-cert\") pod \"kserve-controller-manager-d6f4b65b8-g4zvw\" (UID: \"008f3b93-2fa3-4193-9eb5-c49b118edf92\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"
Jul 10 03:40:51.983915 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:40:51.983895    2571 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 10 03:40:51.983993 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:40:51.983982    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/008f3b93-2fa3-4193-9eb5-c49b118edf92-cert podName:008f3b93-2fa3-4193-9eb5-c49b118edf92 nodeName:}" failed. No retries permitted until 2026-07-10 03:40:52.983960736 +0000 UTC m=+488.498696022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/008f3b93-2fa3-4193-9eb5-c49b118edf92-cert") pod "kserve-controller-manager-d6f4b65b8-g4zvw" (UID: "008f3b93-2fa3-4193-9eb5-c49b118edf92") : secret "kserve-webhook-server-cert" not found
Jul 10 03:40:52.022628 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:52.022599    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg"]
Jul 10 03:40:52.026142 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:40:52.026114    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod51022770_1212_4d2f_abed_3853fc50624c.slice/crio-e4d4156a985b1f8becfba2d8b722fee5055b1140d60697519463c54c7d8dd901 WatchSource:0}: Error finding container e4d4156a985b1f8becfba2d8b722fee5055b1140d60697519463c54c7d8dd901: Status 404 returned error can't find the container with id e4d4156a985b1f8becfba2d8b722fee5055b1140d60697519463c54c7d8dd901
Jul 10 03:40:52.822713 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:52.822678    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg" event={"ID":"51022770-1212-4d2f-abed-3853fc50624c","Type":"ContainerStarted","Data":"e4d4156a985b1f8becfba2d8b722fee5055b1140d60697519463c54c7d8dd901"}
Jul 10 03:40:52.869154 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:52.869119    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"]
Jul 10 03:40:52.869327 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:52.869307    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw" podUID="794f1d63-0711-425b-a327-32d4944ce547" containerName="discovery" containerID="cri-o://a8c6dfd628ea3076f3692c3e552c24f6d50be7f8251f226d63df7b98cf97f5a3" gracePeriod=30
Jul 10 03:40:52.993269 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:52.993233    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/008f3b93-2fa3-4193-9eb5-c49b118edf92-cert\") pod \"kserve-controller-manager-d6f4b65b8-g4zvw\" (UID: \"008f3b93-2fa3-4193-9eb5-c49b118edf92\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"
Jul 10 03:40:52.995551 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:52.995515    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/008f3b93-2fa3-4193-9eb5-c49b118edf92-cert\") pod \"kserve-controller-manager-d6f4b65b8-g4zvw\" (UID: \"008f3b93-2fa3-4193-9eb5-c49b118edf92\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"
Jul 10 03:40:53.090257 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.090223    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"
Jul 10 03:40:53.212908 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.212883    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:53.296385 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.296352    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvcnj\" (UniqueName: \"kubernetes.io/projected/794f1d63-0711-425b-a327-32d4944ce547-kube-api-access-vvcnj\") pod \"794f1d63-0711-425b-a327-32d4944ce547\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") "
Jul 10 03:40:53.296598 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.296416    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-istio-kubeconfig\") pod \"794f1d63-0711-425b-a327-32d4944ce547\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") "
Jul 10 03:40:53.296598 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.296483    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/794f1d63-0711-425b-a327-32d4944ce547-istio-csr-ca-configmap\") pod \"794f1d63-0711-425b-a327-32d4944ce547\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") "
Jul 10 03:40:53.296598 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.296530    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-istio-csr-dns-cert\") pod \"794f1d63-0711-425b-a327-32d4944ce547\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") "
Jul 10 03:40:53.296598 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.296580    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/794f1d63-0711-425b-a327-32d4944ce547-istio-token\") pod \"794f1d63-0711-425b-a327-32d4944ce547\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") "
Jul 10 03:40:53.296814 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.296630    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-cacerts\") pod \"794f1d63-0711-425b-a327-32d4944ce547\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") "
Jul 10 03:40:53.296814 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.296666    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/794f1d63-0711-425b-a327-32d4944ce547-local-certs\") pod \"794f1d63-0711-425b-a327-32d4944ce547\" (UID: \"794f1d63-0711-425b-a327-32d4944ce547\") "
Jul 10 03:40:53.297450 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.297407    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/794f1d63-0711-425b-a327-32d4944ce547-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "794f1d63-0711-425b-a327-32d4944ce547" (UID: "794f1d63-0711-425b-a327-32d4944ce547"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:40:53.299373 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.299332    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "794f1d63-0711-425b-a327-32d4944ce547" (UID: "794f1d63-0711-425b-a327-32d4944ce547"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:40:53.299504 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.299414    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "794f1d63-0711-425b-a327-32d4944ce547" (UID: "794f1d63-0711-425b-a327-32d4944ce547"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:40:53.299504 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.299414    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-cacerts" (OuterVolumeSpecName: "cacerts") pod "794f1d63-0711-425b-a327-32d4944ce547" (UID: "794f1d63-0711-425b-a327-32d4944ce547"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 03:40:53.299629 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.299604    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/794f1d63-0711-425b-a327-32d4944ce547-kube-api-access-vvcnj" (OuterVolumeSpecName: "kube-api-access-vvcnj") pod "794f1d63-0711-425b-a327-32d4944ce547" (UID: "794f1d63-0711-425b-a327-32d4944ce547"). InnerVolumeSpecName "kube-api-access-vvcnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:40:53.299839 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.299810    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/794f1d63-0711-425b-a327-32d4944ce547-istio-token" (OuterVolumeSpecName: "istio-token") pod "794f1d63-0711-425b-a327-32d4944ce547" (UID: "794f1d63-0711-425b-a327-32d4944ce547"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:40:53.300300 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.300283    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794f1d63-0711-425b-a327-32d4944ce547-local-certs" (OuterVolumeSpecName: "local-certs") pod "794f1d63-0711-425b-a327-32d4944ce547" (UID: "794f1d63-0711-425b-a327-32d4944ce547"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:40:53.397314 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.397281    2571 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/794f1d63-0711-425b-a327-32d4944ce547-local-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:40:53.397314 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.397309    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvcnj\" (UniqueName: \"kubernetes.io/projected/794f1d63-0711-425b-a327-32d4944ce547-kube-api-access-vvcnj\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:40:53.397314 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.397319    2571 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-istio-kubeconfig\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:40:53.397556 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.397329    2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/794f1d63-0711-425b-a327-32d4944ce547-istio-csr-ca-configmap\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:40:53.397556 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.397338    2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-istio-csr-dns-cert\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:40:53.397556 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.397346    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/794f1d63-0711-425b-a327-32d4944ce547-istio-token\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:40:53.397556 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.397354    2571 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/794f1d63-0711-425b-a327-32d4944ce547-cacerts\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:40:53.406043 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.406022    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"]
Jul 10 03:40:53.408219 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:40:53.408186    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod008f3b93_2fa3_4193_9eb5_c49b118edf92.slice/crio-81bea7d4ed1455c74df78c0700865eadb9550b1f50d3b8177d9adfcfcb0ac09c WatchSource:0}: Error finding container 81bea7d4ed1455c74df78c0700865eadb9550b1f50d3b8177d9adfcfcb0ac09c: Status 404 returned error can't find the container with id 81bea7d4ed1455c74df78c0700865eadb9550b1f50d3b8177d9adfcfcb0ac09c
Jul 10 03:40:53.829739 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.829704    2571 generic.go:356] "Generic (PLEG): container finished" podID="794f1d63-0711-425b-a327-32d4944ce547" containerID="a8c6dfd628ea3076f3692c3e552c24f6d50be7f8251f226d63df7b98cf97f5a3" exitCode=0
Jul 10 03:40:53.829918 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.829766    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"
Jul 10 03:40:53.829918 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.829798    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw" event={"ID":"794f1d63-0711-425b-a327-32d4944ce547","Type":"ContainerDied","Data":"a8c6dfd628ea3076f3692c3e552c24f6d50be7f8251f226d63df7b98cf97f5a3"}
Jul 10 03:40:53.829918 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.829855    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jfljw" event={"ID":"794f1d63-0711-425b-a327-32d4944ce547","Type":"ContainerDied","Data":"a6b55b8ff59ca8db6158d0498a33a9857833c42d2a16cb31fcd5ace03084e739"}
Jul 10 03:40:53.829918 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.829884    2571 scope.go:117] "RemoveContainer" containerID="a8c6dfd628ea3076f3692c3e552c24f6d50be7f8251f226d63df7b98cf97f5a3"
Jul 10 03:40:53.831591 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.831556    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw" event={"ID":"008f3b93-2fa3-4193-9eb5-c49b118edf92","Type":"ContainerStarted","Data":"81bea7d4ed1455c74df78c0700865eadb9550b1f50d3b8177d9adfcfcb0ac09c"}
Jul 10 03:40:53.839791 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.839767    2571 scope.go:117] "RemoveContainer" containerID="a8c6dfd628ea3076f3692c3e552c24f6d50be7f8251f226d63df7b98cf97f5a3"
Jul 10 03:40:53.840106 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:40:53.840085    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8c6dfd628ea3076f3692c3e552c24f6d50be7f8251f226d63df7b98cf97f5a3\": container with ID starting with a8c6dfd628ea3076f3692c3e552c24f6d50be7f8251f226d63df7b98cf97f5a3 not found: ID does not exist" containerID="a8c6dfd628ea3076f3692c3e552c24f6d50be7f8251f226d63df7b98cf97f5a3"
Jul 10 03:40:53.840172 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.840116    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8c6dfd628ea3076f3692c3e552c24f6d50be7f8251f226d63df7b98cf97f5a3"} err="failed to get container status \"a8c6dfd628ea3076f3692c3e552c24f6d50be7f8251f226d63df7b98cf97f5a3\": rpc error: code = NotFound desc = could not find container \"a8c6dfd628ea3076f3692c3e552c24f6d50be7f8251f226d63df7b98cf97f5a3\": container with ID starting with a8c6dfd628ea3076f3692c3e552c24f6d50be7f8251f226d63df7b98cf97f5a3 not found: ID does not exist"
Jul 10 03:40:53.852209 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.852186    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"]
Jul 10 03:40:53.856329 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:53.856307    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jfljw"]
Jul 10 03:40:55.127712 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:55.127674    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="794f1d63-0711-425b-a327-32d4944ce547" path="/var/lib/kubelet/pods/794f1d63-0711-425b-a327-32d4944ce547/volumes"
Jul 10 03:40:55.791618 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:55.791584    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-mg5lt"
Jul 10 03:40:55.794585 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:55.794560    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-sxbh5"
Jul 10 03:40:56.678257 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:56.678214    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 03:40:56.678624 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:56.678310    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 03:40:56.678624 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:56.678367    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 03:40:56.855754 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:56.855709    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm" event={"ID":"6e0815c6-dc6d-4a74-8621-fbe550124c15","Type":"ContainerStarted","Data":"bf638e3331041448e4ebce1ad120432bfc1b16958fcfc28e52d018fd20187ab2"}
Jul 10 03:40:56.857495 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:56.857375    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw" event={"ID":"008f3b93-2fa3-4193-9eb5-c49b118edf92","Type":"ContainerStarted","Data":"e0ea1a9d9637766058dfeaa768f54694f156e2297fc952ab247c79118d7f6fa3"}
Jul 10 03:40:56.857495 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:56.857444    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"
Jul 10 03:40:56.859278 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:56.859252    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg" event={"ID":"51022770-1212-4d2f-abed-3853fc50624c","Type":"ContainerStarted","Data":"b4216c389803c6988bc9020278e578f56224a4324a889a073acd490c10cc4495"}
Jul 10 03:40:56.859453 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:56.859441    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg"
Jul 10 03:40:56.879568 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:56.879492    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm" podStartSLOduration=1.19807848 podStartE2EDuration="7.879438017s" totalImagesPullingTime="6.681359537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:40:56.878962542 +0000 UTC m=+492.393697848" watchObservedRunningTime="2026-07-10 03:40:56.879438017 +0000 UTC m=+492.394173321"
Jul 10 03:40:56.897097 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:56.897033    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw" podStartSLOduration=2.624537891 podStartE2EDuration="5.897012139s" totalImagesPullingTime="3.272474248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:40:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:40:56.895106532 +0000 UTC m=+492.409841834" watchObservedRunningTime="2026-07-10 03:40:56.897012139 +0000 UTC m=+492.411747441"
Jul 10 03:40:56.912535 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:56.912426    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg" podStartSLOduration=1.258475937 podStartE2EDuration="5.912413524s" totalImagesPullingTime="4.653937587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:40:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:40:56.911173257 +0000 UTC m=+492.425908557" watchObservedRunningTime="2026-07-10 03:40:56.912413524 +0000 UTC m=+492.427148850"
Jul 10 03:40:57.832443 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:57.832407    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:57.837177 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:57.837153    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:57.863582 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:57.863539    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:40:57.864413 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:40:57.864395    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4599pm"
Jul 10 03:41:27.865895 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:27.865866    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9qwfg"
Jul 10 03:41:27.869496 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:27.869478    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g4zvw"
Jul 10 03:41:41.066250 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.066172    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-q44j6"]
Jul 10 03:41:41.066650 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.066518    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="794f1d63-0711-425b-a327-32d4944ce547" containerName="discovery"
Jul 10 03:41:41.066650 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.066529    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="794f1d63-0711-425b-a327-32d4944ce547" containerName="discovery"
Jul 10 03:41:41.066650 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.066591    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="794f1d63-0711-425b-a327-32d4944ce547" containerName="discovery"
Jul 10 03:41:41.068301 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.068286    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-q44j6"
Jul 10 03:41:41.070544 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.070524    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-fv9vt\""
Jul 10 03:41:41.070544 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.070534    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 03:41:41.070725 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.070612    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 03:41:41.077017 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.076989    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-q44j6"]
Jul 10 03:41:41.195980 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.195936    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkx7g\" (UniqueName: \"kubernetes.io/projected/ecedc6bc-2e2a-4d01-b421-23136dac6633-kube-api-access-nkx7g\") pod \"kuadrant-operator-catalog-q44j6\" (UID: \"ecedc6bc-2e2a-4d01-b421-23136dac6633\") " pod="kuadrant-system/kuadrant-operator-catalog-q44j6"
Jul 10 03:41:41.297305 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.297266    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkx7g\" (UniqueName: \"kubernetes.io/projected/ecedc6bc-2e2a-4d01-b421-23136dac6633-kube-api-access-nkx7g\") pod \"kuadrant-operator-catalog-q44j6\" (UID: \"ecedc6bc-2e2a-4d01-b421-23136dac6633\") " pod="kuadrant-system/kuadrant-operator-catalog-q44j6"
Jul 10 03:41:41.305422 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.305395    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkx7g\" (UniqueName: \"kubernetes.io/projected/ecedc6bc-2e2a-4d01-b421-23136dac6633-kube-api-access-nkx7g\") pod \"kuadrant-operator-catalog-q44j6\" (UID: \"ecedc6bc-2e2a-4d01-b421-23136dac6633\") " pod="kuadrant-system/kuadrant-operator-catalog-q44j6"
Jul 10 03:41:41.379446 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.379356    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-q44j6"
Jul 10 03:41:41.436132 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.436093    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-q44j6"]
Jul 10 03:41:41.505416 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.505394    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-q44j6"]
Jul 10 03:41:41.508063 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:41:41.508040    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecedc6bc_2e2a_4d01_b421_23136dac6633.slice/crio-124f119ebc43a80ba197e893be7a23e88f63ce04fffabfada60225e3fde79771 WatchSource:0}: Error finding container 124f119ebc43a80ba197e893be7a23e88f63ce04fffabfada60225e3fde79771: Status 404 returned error can't find the container with id 124f119ebc43a80ba197e893be7a23e88f63ce04fffabfada60225e3fde79771
Jul 10 03:41:41.645065 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.644978    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-57rgw"]
Jul 10 03:41:41.647520 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.647504    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-57rgw"
Jul 10 03:41:41.655187 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.655162    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-57rgw"]
Jul 10 03:41:41.801816 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.801786    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl27j\" (UniqueName: \"kubernetes.io/projected/84bf255d-1361-42a7-91da-eda7f9af1c97-kube-api-access-pl27j\") pod \"kuadrant-operator-catalog-57rgw\" (UID: \"84bf255d-1361-42a7-91da-eda7f9af1c97\") " pod="kuadrant-system/kuadrant-operator-catalog-57rgw"
Jul 10 03:41:41.902838 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.902764    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl27j\" (UniqueName: \"kubernetes.io/projected/84bf255d-1361-42a7-91da-eda7f9af1c97-kube-api-access-pl27j\") pod \"kuadrant-operator-catalog-57rgw\" (UID: \"84bf255d-1361-42a7-91da-eda7f9af1c97\") " pod="kuadrant-system/kuadrant-operator-catalog-57rgw"
Jul 10 03:41:41.914570 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.914546    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl27j\" (UniqueName: \"kubernetes.io/projected/84bf255d-1361-42a7-91da-eda7f9af1c97-kube-api-access-pl27j\") pod \"kuadrant-operator-catalog-57rgw\" (UID: \"84bf255d-1361-42a7-91da-eda7f9af1c97\") " pod="kuadrant-system/kuadrant-operator-catalog-57rgw"
Jul 10 03:41:41.957596 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:41.957545    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-57rgw"
Jul 10 03:41:42.041813 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:42.041777    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-q44j6" event={"ID":"ecedc6bc-2e2a-4d01-b421-23136dac6633","Type":"ContainerStarted","Data":"124f119ebc43a80ba197e893be7a23e88f63ce04fffabfada60225e3fde79771"}
Jul 10 03:41:42.086066 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:42.085919    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-57rgw"]
Jul 10 03:41:42.088347 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:41:42.088322    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84bf255d_1361_42a7_91da_eda7f9af1c97.slice/crio-02a7dcae591d4a280601deb89c0cffd8936d84ee032aa4e2dc8d39248c193d84 WatchSource:0}: Error finding container 02a7dcae591d4a280601deb89c0cffd8936d84ee032aa4e2dc8d39248c193d84: Status 404 returned error can't find the container with id 02a7dcae591d4a280601deb89c0cffd8936d84ee032aa4e2dc8d39248c193d84
Jul 10 03:41:43.050521 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:43.050488    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-57rgw" event={"ID":"84bf255d-1361-42a7-91da-eda7f9af1c97","Type":"ContainerStarted","Data":"02a7dcae591d4a280601deb89c0cffd8936d84ee032aa4e2dc8d39248c193d84"}
Jul 10 03:41:44.056291 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:44.056251    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-q44j6" event={"ID":"ecedc6bc-2e2a-4d01-b421-23136dac6633","Type":"ContainerStarted","Data":"7c6529bc3b2befae315d0813fbe62e401f4793b5ccd36d9e0f91764d63f01296"}
Jul 10 03:41:44.056780 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:44.056347    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-q44j6" podUID="ecedc6bc-2e2a-4d01-b421-23136dac6633" containerName="registry-server" containerID="cri-o://7c6529bc3b2befae315d0813fbe62e401f4793b5ccd36d9e0f91764d63f01296" gracePeriod=2
Jul 10 03:41:44.057782 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:44.057757    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-57rgw" event={"ID":"84bf255d-1361-42a7-91da-eda7f9af1c97","Type":"ContainerStarted","Data":"3d933b107cf09bedde00fcf97fb39c784f0ced81fbcfff4661b4a5e7b98969aa"}
Jul 10 03:41:44.069647 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:44.069595    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-q44j6" podStartSLOduration=0.9442948 podStartE2EDuration="3.069573435s" totalImagesPullingTime="2.125278635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:41:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:41:44.067848861 +0000 UTC m=+539.582584163" watchObservedRunningTime="2026-07-10 03:41:44.069573435 +0000 UTC m=+539.584308739"
Jul 10 03:41:44.290373 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:44.290338    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-q44j6"
Jul 10 03:41:44.305051 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:44.304993    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-57rgw" podStartSLOduration=1.757100597 podStartE2EDuration="3.304973472s" totalImagesPullingTime="1.547872875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:41:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:41:44.079296671 +0000 UTC m=+539.594031976" watchObservedRunningTime="2026-07-10 03:41:44.304973472 +0000 UTC m=+539.819708773"
Jul 10 03:41:44.426058 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:44.426031    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkx7g\" (UniqueName: \"kubernetes.io/projected/ecedc6bc-2e2a-4d01-b421-23136dac6633-kube-api-access-nkx7g\") pod \"ecedc6bc-2e2a-4d01-b421-23136dac6633\" (UID: \"ecedc6bc-2e2a-4d01-b421-23136dac6633\") "
Jul 10 03:41:44.428129 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:44.428107    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecedc6bc-2e2a-4d01-b421-23136dac6633-kube-api-access-nkx7g" (OuterVolumeSpecName: "kube-api-access-nkx7g") pod "ecedc6bc-2e2a-4d01-b421-23136dac6633" (UID: "ecedc6bc-2e2a-4d01-b421-23136dac6633"). InnerVolumeSpecName "kube-api-access-nkx7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:41:44.527614 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:44.527568    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkx7g\" (UniqueName: \"kubernetes.io/projected/ecedc6bc-2e2a-4d01-b421-23136dac6633-kube-api-access-nkx7g\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:41:45.062867 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:45.062834    2571 generic.go:356] "Generic (PLEG): container finished" podID="ecedc6bc-2e2a-4d01-b421-23136dac6633" containerID="7c6529bc3b2befae315d0813fbe62e401f4793b5ccd36d9e0f91764d63f01296" exitCode=0
Jul 10 03:41:45.063276 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:45.062876    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-q44j6"
Jul 10 03:41:45.063276 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:45.062922    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-q44j6" event={"ID":"ecedc6bc-2e2a-4d01-b421-23136dac6633","Type":"ContainerDied","Data":"7c6529bc3b2befae315d0813fbe62e401f4793b5ccd36d9e0f91764d63f01296"}
Jul 10 03:41:45.063276 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:45.062973    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-q44j6" event={"ID":"ecedc6bc-2e2a-4d01-b421-23136dac6633","Type":"ContainerDied","Data":"124f119ebc43a80ba197e893be7a23e88f63ce04fffabfada60225e3fde79771"}
Jul 10 03:41:45.063276 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:45.062998    2571 scope.go:117] "RemoveContainer" containerID="7c6529bc3b2befae315d0813fbe62e401f4793b5ccd36d9e0f91764d63f01296"
Jul 10 03:41:45.071719 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:45.071705    2571 scope.go:117] "RemoveContainer" containerID="7c6529bc3b2befae315d0813fbe62e401f4793b5ccd36d9e0f91764d63f01296"
Jul 10 03:41:45.071942 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:41:45.071923    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c6529bc3b2befae315d0813fbe62e401f4793b5ccd36d9e0f91764d63f01296\": container with ID starting with 7c6529bc3b2befae315d0813fbe62e401f4793b5ccd36d9e0f91764d63f01296 not found: ID does not exist" containerID="7c6529bc3b2befae315d0813fbe62e401f4793b5ccd36d9e0f91764d63f01296"
Jul 10 03:41:45.071990 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:45.071950    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c6529bc3b2befae315d0813fbe62e401f4793b5ccd36d9e0f91764d63f01296"} err="failed to get container status \"7c6529bc3b2befae315d0813fbe62e401f4793b5ccd36d9e0f91764d63f01296\": rpc error: code = NotFound desc = could not find container \"7c6529bc3b2befae315d0813fbe62e401f4793b5ccd36d9e0f91764d63f01296\": container with ID starting with 7c6529bc3b2befae315d0813fbe62e401f4793b5ccd36d9e0f91764d63f01296 not found: ID does not exist"
Jul 10 03:41:45.081278 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:45.081256    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-q44j6"]
Jul 10 03:41:45.086586 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:45.086554    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-q44j6"]
Jul 10 03:41:45.125673 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:45.125649    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecedc6bc-2e2a-4d01-b421-23136dac6633" path="/var/lib/kubelet/pods/ecedc6bc-2e2a-4d01-b421-23136dac6633/volumes"
Jul 10 03:41:51.958389 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:51.958340    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-57rgw"
Jul 10 03:41:51.958908 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:51.958441    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-57rgw"
Jul 10 03:41:51.980735 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:51.980707    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-57rgw"
Jul 10 03:41:52.116865 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:41:52.116838    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-57rgw"
Jul 10 03:42:12.834474 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:12.834402    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hl7p9"]
Jul 10 03:42:12.834954 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:12.834752    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecedc6bc-2e2a-4d01-b421-23136dac6633" containerName="registry-server"
Jul 10 03:42:12.834954 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:12.834763    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecedc6bc-2e2a-4d01-b421-23136dac6633" containerName="registry-server"
Jul 10 03:42:12.834954 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:12.834816    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecedc6bc-2e2a-4d01-b421-23136dac6633" containerName="registry-server"
Jul 10 03:42:12.838228 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:12.838212    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hl7p9"
Jul 10 03:42:12.840405 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:12.840385    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 10 03:42:12.840405 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:12.840397    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qdkgj\""
Jul 10 03:42:12.845997 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:12.845973    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hl7p9"]
Jul 10 03:42:12.959743 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:12.959705    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkcnh\" (UniqueName: \"kubernetes.io/projected/cb887640-90c8-4e82-8b16-b91764e68298-kube-api-access-hkcnh\") pod \"dns-operator-controller-manager-864dd78c4c-hl7p9\" (UID: \"cb887640-90c8-4e82-8b16-b91764e68298\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hl7p9"
Jul 10 03:42:13.060820 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:13.060780    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkcnh\" (UniqueName: \"kubernetes.io/projected/cb887640-90c8-4e82-8b16-b91764e68298-kube-api-access-hkcnh\") pod \"dns-operator-controller-manager-864dd78c4c-hl7p9\" (UID: \"cb887640-90c8-4e82-8b16-b91764e68298\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hl7p9"
Jul 10 03:42:13.072783 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:13.072747    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkcnh\" (UniqueName: \"kubernetes.io/projected/cb887640-90c8-4e82-8b16-b91764e68298-kube-api-access-hkcnh\") pod \"dns-operator-controller-manager-864dd78c4c-hl7p9\" (UID: \"cb887640-90c8-4e82-8b16-b91764e68298\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hl7p9"
Jul 10 03:42:13.148526 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:13.148414    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hl7p9"
Jul 10 03:42:13.281391 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:13.281351    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hl7p9"]
Jul 10 03:42:13.284766 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:42:13.284738    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb887640_90c8_4e82_8b16_b91764e68298.slice/crio-70366d2ca8709515ad023181b8ff549cc13102df5710f4c46a9f8f0b83d53b02 WatchSource:0}: Error finding container 70366d2ca8709515ad023181b8ff549cc13102df5710f4c46a9f8f0b83d53b02: Status 404 returned error can't find the container with id 70366d2ca8709515ad023181b8ff549cc13102df5710f4c46a9f8f0b83d53b02
Jul 10 03:42:14.188531 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:14.188490    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hl7p9" event={"ID":"cb887640-90c8-4e82-8b16-b91764e68298","Type":"ContainerStarted","Data":"70366d2ca8709515ad023181b8ff549cc13102df5710f4c46a9f8f0b83d53b02"}
Jul 10 03:42:16.197797 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:16.197766    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hl7p9" event={"ID":"cb887640-90c8-4e82-8b16-b91764e68298","Type":"ContainerStarted","Data":"4b441fce71e2960293e406197721d14aef3df98dd7320e7a24df7f7771fa5ed0"}
Jul 10 03:42:16.198179 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:16.197880    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hl7p9"
Jul 10 03:42:16.212666 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:16.212619    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hl7p9" podStartSLOduration=2.27983015 podStartE2EDuration="4.212599315s" totalImagesPullingTime="1.932769165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:42:16.212119589 +0000 UTC m=+571.726854890" watchObservedRunningTime="2026-07-10 03:42:16.212599315 +0000 UTC m=+571.727334637"
Jul 10 03:42:18.119908 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:18.119877    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"]
Jul 10 03:42:18.123340 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:18.123312    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:18.125785 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:18.125764    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vh552\""
Jul 10 03:42:18.140189 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:18.140166    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"]
Jul 10 03:42:18.205602 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:18.205572    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9nw4\" (UniqueName: \"kubernetes.io/projected/9714104d-7263-40de-ad29-2c0a3bd5b478-kube-api-access-b9nw4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" (UID: \"9714104d-7263-40de-ad29-2c0a3bd5b478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:18.205804 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:18.205637    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9714104d-7263-40de-ad29-2c0a3bd5b478-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" (UID: \"9714104d-7263-40de-ad29-2c0a3bd5b478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:18.306849 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:18.306820    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9nw4\" (UniqueName: \"kubernetes.io/projected/9714104d-7263-40de-ad29-2c0a3bd5b478-kube-api-access-b9nw4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" (UID: \"9714104d-7263-40de-ad29-2c0a3bd5b478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:18.307020 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:18.306882    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9714104d-7263-40de-ad29-2c0a3bd5b478-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" (UID: \"9714104d-7263-40de-ad29-2c0a3bd5b478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:18.307216 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:18.307196    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9714104d-7263-40de-ad29-2c0a3bd5b478-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" (UID: \"9714104d-7263-40de-ad29-2c0a3bd5b478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:18.316565 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:18.316529    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9nw4\" (UniqueName: \"kubernetes.io/projected/9714104d-7263-40de-ad29-2c0a3bd5b478-kube-api-access-b9nw4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" (UID: \"9714104d-7263-40de-ad29-2c0a3bd5b478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:18.433962 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:18.433923    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:18.563941 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:18.563917    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"]
Jul 10 03:42:18.566093 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:42:18.566062    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9714104d_7263_40de_ad29_2c0a3bd5b478.slice/crio-8d403b9988b347b08d96b44ad713ba0c5f29089b60d72d120505984f509a892b WatchSource:0}: Error finding container 8d403b9988b347b08d96b44ad713ba0c5f29089b60d72d120505984f509a892b: Status 404 returned error can't find the container with id 8d403b9988b347b08d96b44ad713ba0c5f29089b60d72d120505984f509a892b
Jul 10 03:42:19.213622 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:19.213587    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf" event={"ID":"9714104d-7263-40de-ad29-2c0a3bd5b478","Type":"ContainerStarted","Data":"8d403b9988b347b08d96b44ad713ba0c5f29089b60d72d120505984f509a892b"}
Jul 10 03:42:21.143549 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:21.143517    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"]
Jul 10 03:42:21.147064 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:21.147040    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:21.149114 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:21.149090    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-m8w5t\""
Jul 10 03:42:21.155675 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:21.155642    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"]
Jul 10 03:42:21.235810 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:21.235782    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mxm2\" (UniqueName: \"kubernetes.io/projected/469dbc33-4cb4-45cf-aa6e-858a5d5e6985-kube-api-access-2mxm2\") pod \"limitador-operator-controller-manager-859d7b78bd-t8h9f\" (UID: \"469dbc33-4cb4-45cf-aa6e-858a5d5e6985\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:21.336996 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:21.336961    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mxm2\" (UniqueName: \"kubernetes.io/projected/469dbc33-4cb4-45cf-aa6e-858a5d5e6985-kube-api-access-2mxm2\") pod \"limitador-operator-controller-manager-859d7b78bd-t8h9f\" (UID: \"469dbc33-4cb4-45cf-aa6e-858a5d5e6985\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:21.345453 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:21.345429    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mxm2\" (UniqueName: \"kubernetes.io/projected/469dbc33-4cb4-45cf-aa6e-858a5d5e6985-kube-api-access-2mxm2\") pod \"limitador-operator-controller-manager-859d7b78bd-t8h9f\" (UID: \"469dbc33-4cb4-45cf-aa6e-858a5d5e6985\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:21.460043 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:21.460004    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:21.594735 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:21.594706    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"]
Jul 10 03:42:21.597808 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:42:21.597780    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469dbc33_4cb4_45cf_aa6e_858a5d5e6985.slice/crio-b0feff7ed331d6ea8e7187e2dd97710ddf6f280603504350f7f08283b19dd7cf WatchSource:0}: Error finding container b0feff7ed331d6ea8e7187e2dd97710ddf6f280603504350f7f08283b19dd7cf: Status 404 returned error can't find the container with id b0feff7ed331d6ea8e7187e2dd97710ddf6f280603504350f7f08283b19dd7cf
Jul 10 03:42:22.229645 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:22.229600    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f" event={"ID":"469dbc33-4cb4-45cf-aa6e-858a5d5e6985","Type":"ContainerStarted","Data":"b0feff7ed331d6ea8e7187e2dd97710ddf6f280603504350f7f08283b19dd7cf"}
Jul 10 03:42:24.407910 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:24.407875    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-b2ntp"]
Jul 10 03:42:24.411501 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:24.411481    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b2ntp"
Jul 10 03:42:24.413843 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:24.413816    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-nkstl\""
Jul 10 03:42:24.421865 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:24.421833    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-b2ntp"]
Jul 10 03:42:24.568135 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:24.568093    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbjsp\" (UniqueName: \"kubernetes.io/projected/8885b43a-19d0-4f65-9f22-0b8f3b5aba6a-kube-api-access-sbjsp\") pod \"authorino-operator-75cc8d5cb7-b2ntp\" (UID: \"8885b43a-19d0-4f65-9f22-0b8f3b5aba6a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-b2ntp"
Jul 10 03:42:24.670150 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:24.669057    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbjsp\" (UniqueName: \"kubernetes.io/projected/8885b43a-19d0-4f65-9f22-0b8f3b5aba6a-kube-api-access-sbjsp\") pod \"authorino-operator-75cc8d5cb7-b2ntp\" (UID: \"8885b43a-19d0-4f65-9f22-0b8f3b5aba6a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-b2ntp"
Jul 10 03:42:24.680594 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:24.680558    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbjsp\" (UniqueName: \"kubernetes.io/projected/8885b43a-19d0-4f65-9f22-0b8f3b5aba6a-kube-api-access-sbjsp\") pod \"authorino-operator-75cc8d5cb7-b2ntp\" (UID: \"8885b43a-19d0-4f65-9f22-0b8f3b5aba6a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-b2ntp"
Jul 10 03:42:24.723156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:24.723119    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b2ntp"
Jul 10 03:42:24.862032 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:24.861884    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-b2ntp"]
Jul 10 03:42:24.864777 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:42:24.864747    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8885b43a_19d0_4f65_9f22_0b8f3b5aba6a.slice/crio-061ed67bc28def0d07ca63b1f6a0f5ab62df0790658599e58d80f76163eda985 WatchSource:0}: Error finding container 061ed67bc28def0d07ca63b1f6a0f5ab62df0790658599e58d80f76163eda985: Status 404 returned error can't find the container with id 061ed67bc28def0d07ca63b1f6a0f5ab62df0790658599e58d80f76163eda985
Jul 10 03:42:25.248620 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:25.248582    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf" event={"ID":"9714104d-7263-40de-ad29-2c0a3bd5b478","Type":"ContainerStarted","Data":"f63e5c6be41a24e2e01a607f52e1d923190ec6cb800aad7588c82b940fb4c32c"}
Jul 10 03:42:25.248810 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:25.248656    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:25.249973 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:25.249948    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b2ntp" event={"ID":"8885b43a-19d0-4f65-9f22-0b8f3b5aba6a","Type":"ContainerStarted","Data":"061ed67bc28def0d07ca63b1f6a0f5ab62df0790658599e58d80f76163eda985"}
Jul 10 03:42:25.251564 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:25.251531    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f" event={"ID":"469dbc33-4cb4-45cf-aa6e-858a5d5e6985","Type":"ContainerStarted","Data":"cb0ef5e7f05b7ad3752e5a8383c3e207a5ab35b99bce87ca77696966b550190f"}
Jul 10 03:42:25.251704 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:25.251674    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:25.268910 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:25.268863    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf" podStartSLOduration=1.3810249159999999 podStartE2EDuration="7.268842078s" totalImagesPullingTime="5.887817162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:42:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:42:25.26720002 +0000 UTC m=+580.781935327" watchObservedRunningTime="2026-07-10 03:42:25.268842078 +0000 UTC m=+580.783577379"
Jul 10 03:42:25.287340 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:25.287288    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f" podStartSLOduration=1.4471867330000001 podStartE2EDuration="4.287269706s" totalImagesPullingTime="2.840082973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:42:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:42:25.286089298 +0000 UTC m=+580.800824600" watchObservedRunningTime="2026-07-10 03:42:25.287269706 +0000 UTC m=+580.802005073"
Jul 10 03:42:27.205250 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:27.205223    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hl7p9"
Jul 10 03:42:27.262873 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:27.262763    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b2ntp" event={"ID":"8885b43a-19d0-4f65-9f22-0b8f3b5aba6a","Type":"ContainerStarted","Data":"83ab4fb0ffcad159ba1e0385f5e2995ab9426a0c19a6820d01ffa171b668789a"}
Jul 10 03:42:27.262873 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:27.262862    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b2ntp"
Jul 10 03:42:27.279066 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:27.279012    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b2ntp" podStartSLOduration=1.134135243 podStartE2EDuration="3.278995077s" totalImagesPullingTime="2.144859834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:42:27.276755343 +0000 UTC m=+582.791490644" watchObservedRunningTime="2026-07-10 03:42:27.278995077 +0000 UTC m=+582.793730379"
Jul 10 03:42:36.258280 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:36.258248    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:36.258713 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:36.258325    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:37.996684 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:37.996647    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"]
Jul 10 03:42:37.997117 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:37.996921    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" containerName="manager" containerID="cri-o://f63e5c6be41a24e2e01a607f52e1d923190ec6cb800aad7588c82b940fb4c32c" gracePeriod=2
Jul 10 03:42:37.999260 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:37.999235    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"]
Jul 10 03:42:37.999911 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:37.999865    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:38.017697 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.017663    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"]
Jul 10 03:42:38.018017 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.017966    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" containerName="manager" containerID="cri-o://cb0ef5e7f05b7ad3752e5a8383c3e207a5ab35b99bce87ca77696966b550190f" gracePeriod=2
Jul 10 03:42:38.020666 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.020632    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:38.029080 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.029052    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"]
Jul 10 03:42:38.030795 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.030772    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"]
Jul 10 03:42:38.031131 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.031116    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" containerName="manager"
Jul 10 03:42:38.031131 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.031131    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" containerName="manager"
Jul 10 03:42:38.031256 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.031147    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" containerName="manager"
Jul 10 03:42:38.031256 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.031153    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" containerName="manager"
Jul 10 03:42:38.031256 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.031218    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" containerName="manager"
Jul 10 03:42:38.031256 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.031230    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" containerName="manager"
Jul 10 03:42:38.034196 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.034177    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"
Jul 10 03:42:38.052327 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.052260    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlshf"]
Jul 10 03:42:38.062414 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.062312    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"]
Jul 10 03:42:38.062414 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.062331    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlshf"
Jul 10 03:42:38.068557 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.068503    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:38.070003 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.069983    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlshf"]
Jul 10 03:42:38.070614 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.070591    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-t8h9f\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:38.095765 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.095439    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:38.097430 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.097404    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-t8h9f\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:38.182835 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.182793    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38c05155-aa6c-4454-847c-a12932394184-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5gbt7\" (UID: \"38c05155-aa6c-4454-847c-a12932394184\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"
Jul 10 03:42:38.182835 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.182840    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94nhm\" (UniqueName: \"kubernetes.io/projected/38c05155-aa6c-4454-847c-a12932394184-kube-api-access-94nhm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5gbt7\" (UID: \"38c05155-aa6c-4454-847c-a12932394184\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"
Jul 10 03:42:38.183038 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.182919    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqnh5\" (UniqueName: \"kubernetes.io/projected/b00144cf-d55c-4961-8607-d76ed6f17139-kube-api-access-lqnh5\") pod \"limitador-operator-controller-manager-859d7b78bd-vlshf\" (UID: \"b00144cf-d55c-4961-8607-d76ed6f17139\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlshf"
Jul 10 03:42:38.259925 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.259877    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:38.261945 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.261918    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:38.262663 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.262642    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:38.263630 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.263597    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-t8h9f\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:38.265224 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.265199    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:38.266865 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.266839    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-t8h9f\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:38.269689 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.269673    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-b2ntp"
Jul 10 03:42:38.271779 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.271760    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:38.273828 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.273810    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-t8h9f\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:38.284082 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.284060    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqnh5\" (UniqueName: \"kubernetes.io/projected/b00144cf-d55c-4961-8607-d76ed6f17139-kube-api-access-lqnh5\") pod \"limitador-operator-controller-manager-859d7b78bd-vlshf\" (UID: \"b00144cf-d55c-4961-8607-d76ed6f17139\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlshf"
Jul 10 03:42:38.284163 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.284119    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38c05155-aa6c-4454-847c-a12932394184-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5gbt7\" (UID: \"38c05155-aa6c-4454-847c-a12932394184\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"
Jul 10 03:42:38.284163 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.284139    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94nhm\" (UniqueName: \"kubernetes.io/projected/38c05155-aa6c-4454-847c-a12932394184-kube-api-access-94nhm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5gbt7\" (UID: \"38c05155-aa6c-4454-847c-a12932394184\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"
Jul 10 03:42:38.284583 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.284562    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38c05155-aa6c-4454-847c-a12932394184-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5gbt7\" (UID: \"38c05155-aa6c-4454-847c-a12932394184\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"
Jul 10 03:42:38.298385 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.298351    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqnh5\" (UniqueName: \"kubernetes.io/projected/b00144cf-d55c-4961-8607-d76ed6f17139-kube-api-access-lqnh5\") pod \"limitador-operator-controller-manager-859d7b78bd-vlshf\" (UID: \"b00144cf-d55c-4961-8607-d76ed6f17139\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlshf"
Jul 10 03:42:38.298515 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.298361    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94nhm\" (UniqueName: \"kubernetes.io/projected/38c05155-aa6c-4454-847c-a12932394184-kube-api-access-94nhm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5gbt7\" (UID: \"38c05155-aa6c-4454-847c-a12932394184\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"
Jul 10 03:42:38.300500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.300481    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"]
Jul 10 03:42:38.303528 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.303514    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"
Jul 10 03:42:38.305988 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.305964    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:38.307765 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.307577    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-t8h9f\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:38.314195 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.314167    2571 generic.go:356] "Generic (PLEG): container finished" podID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" containerID="cb0ef5e7f05b7ad3752e5a8383c3e207a5ab35b99bce87ca77696966b550190f" exitCode=0
Jul 10 03:42:38.314314 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.314299    2571 scope.go:117] "RemoveContainer" containerID="cb0ef5e7f05b7ad3752e5a8383c3e207a5ab35b99bce87ca77696966b550190f"
Jul 10 03:42:38.314442 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.314407    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:38.316085 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.316009    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"]
Jul 10 03:42:38.317573 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.317259    2571 generic.go:356] "Generic (PLEG): container finished" podID="9714104d-7263-40de-ad29-2c0a3bd5b478" containerID="f63e5c6be41a24e2e01a607f52e1d923190ec6cb800aad7588c82b940fb4c32c" exitCode=0
Jul 10 03:42:38.317573 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.317405    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:38.324932 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.324910    2571 scope.go:117] "RemoveContainer" containerID="cb0ef5e7f05b7ad3752e5a8383c3e207a5ab35b99bce87ca77696966b550190f"
Jul 10 03:42:38.325254 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.325228    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb0ef5e7f05b7ad3752e5a8383c3e207a5ab35b99bce87ca77696966b550190f\": container with ID starting with cb0ef5e7f05b7ad3752e5a8383c3e207a5ab35b99bce87ca77696966b550190f not found: ID does not exist" containerID="cb0ef5e7f05b7ad3752e5a8383c3e207a5ab35b99bce87ca77696966b550190f"
Jul 10 03:42:38.325350 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.325264    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb0ef5e7f05b7ad3752e5a8383c3e207a5ab35b99bce87ca77696966b550190f"} err="failed to get container status \"cb0ef5e7f05b7ad3752e5a8383c3e207a5ab35b99bce87ca77696966b550190f\": rpc error: code = NotFound desc = could not find container \"cb0ef5e7f05b7ad3752e5a8383c3e207a5ab35b99bce87ca77696966b550190f\": container with ID starting with cb0ef5e7f05b7ad3752e5a8383c3e207a5ab35b99bce87ca77696966b550190f not found: ID does not exist"
Jul 10 03:42:38.325350 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.325289    2571 scope.go:117] "RemoveContainer" containerID="f63e5c6be41a24e2e01a607f52e1d923190ec6cb800aad7588c82b940fb4c32c"
Jul 10 03:42:38.333676 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.333656    2571 scope.go:117] "RemoveContainer" containerID="f63e5c6be41a24e2e01a607f52e1d923190ec6cb800aad7588c82b940fb4c32c"
Jul 10 03:42:38.333956 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.333937    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f63e5c6be41a24e2e01a607f52e1d923190ec6cb800aad7588c82b940fb4c32c\": container with ID starting with f63e5c6be41a24e2e01a607f52e1d923190ec6cb800aad7588c82b940fb4c32c not found: ID does not exist" containerID="f63e5c6be41a24e2e01a607f52e1d923190ec6cb800aad7588c82b940fb4c32c"
Jul 10 03:42:38.334035 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.333962    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f63e5c6be41a24e2e01a607f52e1d923190ec6cb800aad7588c82b940fb4c32c"} err="failed to get container status \"f63e5c6be41a24e2e01a607f52e1d923190ec6cb800aad7588c82b940fb4c32c\": rpc error: code = NotFound desc = could not find container \"f63e5c6be41a24e2e01a607f52e1d923190ec6cb800aad7588c82b940fb4c32c\": container with ID starting with f63e5c6be41a24e2e01a607f52e1d923190ec6cb800aad7588c82b940fb4c32c not found: ID does not exist"
Jul 10 03:42:38.334408 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.334386    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:38.336175 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.336153    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-t8h9f\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:38.384655 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.384616    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mxm2\" (UniqueName: \"kubernetes.io/projected/469dbc33-4cb4-45cf-aa6e-858a5d5e6985-kube-api-access-2mxm2\") pod \"469dbc33-4cb4-45cf-aa6e-858a5d5e6985\" (UID: \"469dbc33-4cb4-45cf-aa6e-858a5d5e6985\") "
Jul 10 03:42:38.384655 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.384658    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9714104d-7263-40de-ad29-2c0a3bd5b478-extensions-socket-volume\") pod \"9714104d-7263-40de-ad29-2c0a3bd5b478\" (UID: \"9714104d-7263-40de-ad29-2c0a3bd5b478\") "
Jul 10 03:42:38.384861 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.384687    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9nw4\" (UniqueName: \"kubernetes.io/projected/9714104d-7263-40de-ad29-2c0a3bd5b478-kube-api-access-b9nw4\") pod \"9714104d-7263-40de-ad29-2c0a3bd5b478\" (UID: \"9714104d-7263-40de-ad29-2c0a3bd5b478\") "
Jul 10 03:42:38.384861 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.384823    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e03b04f6-7657-4483-9ee8-eefc2c92749b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w9xdf\" (UID: \"e03b04f6-7657-4483-9ee8-eefc2c92749b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"
Jul 10 03:42:38.384954 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.384867    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wtz2\" (UniqueName: \"kubernetes.io/projected/e03b04f6-7657-4483-9ee8-eefc2c92749b-kube-api-access-8wtz2\") pod \"kuadrant-operator-controller-manager-86878c769f-w9xdf\" (UID: \"e03b04f6-7657-4483-9ee8-eefc2c92749b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"
Jul 10 03:42:38.385148 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.385116    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9714104d-7263-40de-ad29-2c0a3bd5b478-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9714104d-7263-40de-ad29-2c0a3bd5b478" (UID: "9714104d-7263-40de-ad29-2c0a3bd5b478"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:42:38.386613 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.386592    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/469dbc33-4cb4-45cf-aa6e-858a5d5e6985-kube-api-access-2mxm2" (OuterVolumeSpecName: "kube-api-access-2mxm2") pod "469dbc33-4cb4-45cf-aa6e-858a5d5e6985" (UID: "469dbc33-4cb4-45cf-aa6e-858a5d5e6985"). InnerVolumeSpecName "kube-api-access-2mxm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:42:38.386681 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.386624    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9714104d-7263-40de-ad29-2c0a3bd5b478-kube-api-access-b9nw4" (OuterVolumeSpecName: "kube-api-access-b9nw4") pod "9714104d-7263-40de-ad29-2c0a3bd5b478" (UID: "9714104d-7263-40de-ad29-2c0a3bd5b478"). InnerVolumeSpecName "kube-api-access-b9nw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:42:38.415519 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.415484    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"
Jul 10 03:42:38.421191 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.421174    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlshf"
Jul 10 03:42:38.485968 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.485494    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e03b04f6-7657-4483-9ee8-eefc2c92749b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w9xdf\" (UID: \"e03b04f6-7657-4483-9ee8-eefc2c92749b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"
Jul 10 03:42:38.485968 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.485535    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wtz2\" (UniqueName: \"kubernetes.io/projected/e03b04f6-7657-4483-9ee8-eefc2c92749b-kube-api-access-8wtz2\") pod \"kuadrant-operator-controller-manager-86878c769f-w9xdf\" (UID: \"e03b04f6-7657-4483-9ee8-eefc2c92749b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"
Jul 10 03:42:38.485968 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.485605    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mxm2\" (UniqueName: \"kubernetes.io/projected/469dbc33-4cb4-45cf-aa6e-858a5d5e6985-kube-api-access-2mxm2\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:42:38.485968 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.485624    2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9714104d-7263-40de-ad29-2c0a3bd5b478-extensions-socket-volume\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:42:38.485968 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.485639    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9nw4\" (UniqueName: \"kubernetes.io/projected/9714104d-7263-40de-ad29-2c0a3bd5b478-kube-api-access-b9nw4\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:42:38.485968 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.485896    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e03b04f6-7657-4483-9ee8-eefc2c92749b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w9xdf\" (UID: \"e03b04f6-7657-4483-9ee8-eefc2c92749b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"
Jul 10 03:42:38.499694 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.499626    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wtz2\" (UniqueName: \"kubernetes.io/projected/e03b04f6-7657-4483-9ee8-eefc2c92749b-kube-api-access-8wtz2\") pod \"kuadrant-operator-controller-manager-86878c769f-w9xdf\" (UID: \"e03b04f6-7657-4483-9ee8-eefc2c92749b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"
Jul 10 03:42:38.565493 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.565449    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"]
Jul 10 03:42:38.567665 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:42:38.567621    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38c05155_aa6c_4454_847c_a12932394184.slice/crio-eebe2f47c05c64fe4cbf1fd98f2143124e357828c1a5ec559f65dec75a5c2eae WatchSource:0}: Error finding container eebe2f47c05c64fe4cbf1fd98f2143124e357828c1a5ec559f65dec75a5c2eae: Status 404 returned error can't find the container with id eebe2f47c05c64fe4cbf1fd98f2143124e357828c1a5ec559f65dec75a5c2eae
Jul 10 03:42:38.589155 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.589127    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlshf"]
Jul 10 03:42:38.590764 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:42:38.590733    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb00144cf_d55c_4961_8607_d76ed6f17139.slice/crio-b905c90ad9a625bd5b6df6916be4f9c318618360b3fc2a3ee82e1a2bfe51cb67 WatchSource:0}: Error finding container b905c90ad9a625bd5b6df6916be4f9c318618360b3fc2a3ee82e1a2bfe51cb67: Status 404 returned error can't find the container with id b905c90ad9a625bd5b6df6916be4f9c318618360b3fc2a3ee82e1a2bfe51cb67
Jul 10 03:42:38.615275 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.615245    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"
Jul 10 03:42:38.639373 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.639344    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:38.642051 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.642022    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-t8h9f\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:38.643925 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.643898    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9jfwf\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9jfwf"
Jul 10 03:42:38.645730 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:38.645625    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-t8h9f\" is forbidden: User \"system:node:ip-10-0-143-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-142.ec2.internal' and this object" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-t8h9f"
Jul 10 03:42:38.763578 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:38.763496    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"]
Jul 10 03:42:38.767964 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:42:38.767938    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode03b04f6_7657_4483_9ee8_eefc2c92749b.slice/crio-4d4d8d525fd1b3f57d4e0f8c8f0cfb3edfe263e17f2b3b649492e34706a15313 WatchSource:0}: Error finding container 4d4d8d525fd1b3f57d4e0f8c8f0cfb3edfe263e17f2b3b649492e34706a15313: Status 404 returned error can't find the container with id 4d4d8d525fd1b3f57d4e0f8c8f0cfb3edfe263e17f2b3b649492e34706a15313
Jul 10 03:42:39.127724 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.127687    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="469dbc33-4cb4-45cf-aa6e-858a5d5e6985" path="/var/lib/kubelet/pods/469dbc33-4cb4-45cf-aa6e-858a5d5e6985/volumes"
Jul 10 03:42:39.128198 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.128178    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9714104d-7263-40de-ad29-2c0a3bd5b478" path="/var/lib/kubelet/pods/9714104d-7263-40de-ad29-2c0a3bd5b478/volumes"
Jul 10 03:42:39.325389 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.325306    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7" event={"ID":"38c05155-aa6c-4454-847c-a12932394184","Type":"ContainerStarted","Data":"f0caf5b618fe2940fda5d0b03d2869221c6ce498c2fd4128e9ab105e6445e823"}
Jul 10 03:42:39.325389 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.325370    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"
Jul 10 03:42:39.325613 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.325393    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7" event={"ID":"38c05155-aa6c-4454-847c-a12932394184","Type":"ContainerStarted","Data":"eebe2f47c05c64fe4cbf1fd98f2143124e357828c1a5ec559f65dec75a5c2eae"}
Jul 10 03:42:39.326969 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.326937    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf" event={"ID":"e03b04f6-7657-4483-9ee8-eefc2c92749b","Type":"ContainerStarted","Data":"46fc8c220efd08b1c25e21676f95f639164aa41c4087af994f7b11f9078d7cca"}
Jul 10 03:42:39.327095 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.326988    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf" event={"ID":"e03b04f6-7657-4483-9ee8-eefc2c92749b","Type":"ContainerStarted","Data":"4d4d8d525fd1b3f57d4e0f8c8f0cfb3edfe263e17f2b3b649492e34706a15313"}
Jul 10 03:42:39.327147 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.327089    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"
Jul 10 03:42:39.330285 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.330266    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlshf" event={"ID":"b00144cf-d55c-4961-8607-d76ed6f17139","Type":"ContainerStarted","Data":"eb31eca493ebf37e88d8eabf0a274c6cbeeefca6ff2dda98faf61a1a074cf252"}
Jul 10 03:42:39.330351 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.330298    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlshf" event={"ID":"b00144cf-d55c-4961-8607-d76ed6f17139","Type":"ContainerStarted","Data":"b905c90ad9a625bd5b6df6916be4f9c318618360b3fc2a3ee82e1a2bfe51cb67"}
Jul 10 03:42:39.330351 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.330323    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlshf"
Jul 10 03:42:39.349405 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.349362    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7" podStartSLOduration=1.349348947 podStartE2EDuration="1.349348947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:42:39.347344922 +0000 UTC m=+594.862080226" watchObservedRunningTime="2026-07-10 03:42:39.349348947 +0000 UTC m=+594.864084248"
Jul 10 03:42:39.364130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.364082    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf" podStartSLOduration=1.3640652389999999 podStartE2EDuration="1.364065239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:42:39.363641607 +0000 UTC m=+594.878376943" watchObservedRunningTime="2026-07-10 03:42:39.364065239 +0000 UTC m=+594.878800543"
Jul 10 03:42:39.379912 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:39.379868    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlshf" podStartSLOduration=1.37985207 podStartE2EDuration="1.37985207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:42:39.378093814 +0000 UTC m=+594.892829116" watchObservedRunningTime="2026-07-10 03:42:39.37985207 +0000 UTC m=+594.894587427"
Jul 10 03:42:50.337413 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:50.337383    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"
Jul 10 03:42:50.343250 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:50.337451    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vlshf"
Jul 10 03:42:50.343250 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:50.337631    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"
Jul 10 03:42:50.388167 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:50.388131    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"]
Jul 10 03:42:50.388385 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:50.388360    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7" podUID="38c05155-aa6c-4454-847c-a12932394184" containerName="manager" containerID="cri-o://f0caf5b618fe2940fda5d0b03d2869221c6ce498c2fd4128e9ab105e6445e823" gracePeriod=10
Jul 10 03:42:50.648055 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:50.648030    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"
Jul 10 03:42:50.670174 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:50.670150    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38c05155-aa6c-4454-847c-a12932394184-extensions-socket-volume\") pod \"38c05155-aa6c-4454-847c-a12932394184\" (UID: \"38c05155-aa6c-4454-847c-a12932394184\") "
Jul 10 03:42:50.670317 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:50.670182    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94nhm\" (UniqueName: \"kubernetes.io/projected/38c05155-aa6c-4454-847c-a12932394184-kube-api-access-94nhm\") pod \"38c05155-aa6c-4454-847c-a12932394184\" (UID: \"38c05155-aa6c-4454-847c-a12932394184\") "
Jul 10 03:42:50.670548 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:50.670524    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c05155-aa6c-4454-847c-a12932394184-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "38c05155-aa6c-4454-847c-a12932394184" (UID: "38c05155-aa6c-4454-847c-a12932394184"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:42:50.672768 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:50.672738    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38c05155-aa6c-4454-847c-a12932394184-kube-api-access-94nhm" (OuterVolumeSpecName: "kube-api-access-94nhm") pod "38c05155-aa6c-4454-847c-a12932394184" (UID: "38c05155-aa6c-4454-847c-a12932394184"). InnerVolumeSpecName "kube-api-access-94nhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:42:50.771288 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:50.771252    2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/38c05155-aa6c-4454-847c-a12932394184-extensions-socket-volume\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:42:50.771288 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:50.771288    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94nhm\" (UniqueName: \"kubernetes.io/projected/38c05155-aa6c-4454-847c-a12932394184-kube-api-access-94nhm\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:42:51.387823 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:51.387740    2571 generic.go:356] "Generic (PLEG): container finished" podID="38c05155-aa6c-4454-847c-a12932394184" containerID="f0caf5b618fe2940fda5d0b03d2869221c6ce498c2fd4128e9ab105e6445e823" exitCode=0
Jul 10 03:42:51.387823 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:51.387784    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"
Jul 10 03:42:51.388226 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:51.387785    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7" event={"ID":"38c05155-aa6c-4454-847c-a12932394184","Type":"ContainerDied","Data":"f0caf5b618fe2940fda5d0b03d2869221c6ce498c2fd4128e9ab105e6445e823"}
Jul 10 03:42:51.388226 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:51.387910    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7" event={"ID":"38c05155-aa6c-4454-847c-a12932394184","Type":"ContainerDied","Data":"eebe2f47c05c64fe4cbf1fd98f2143124e357828c1a5ec559f65dec75a5c2eae"}
Jul 10 03:42:51.388226 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:51.387934    2571 scope.go:117] "RemoveContainer" containerID="f0caf5b618fe2940fda5d0b03d2869221c6ce498c2fd4128e9ab105e6445e823"
Jul 10 03:42:51.396201 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:51.396182    2571 scope.go:117] "RemoveContainer" containerID="f0caf5b618fe2940fda5d0b03d2869221c6ce498c2fd4128e9ab105e6445e823"
Jul 10 03:42:51.396478 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:42:51.396439    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0caf5b618fe2940fda5d0b03d2869221c6ce498c2fd4128e9ab105e6445e823\": container with ID starting with f0caf5b618fe2940fda5d0b03d2869221c6ce498c2fd4128e9ab105e6445e823 not found: ID does not exist" containerID="f0caf5b618fe2940fda5d0b03d2869221c6ce498c2fd4128e9ab105e6445e823"
Jul 10 03:42:51.396531 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:51.396485    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0caf5b618fe2940fda5d0b03d2869221c6ce498c2fd4128e9ab105e6445e823"} err="failed to get container status \"f0caf5b618fe2940fda5d0b03d2869221c6ce498c2fd4128e9ab105e6445e823\": rpc error: code = NotFound desc = could not find container \"f0caf5b618fe2940fda5d0b03d2869221c6ce498c2fd4128e9ab105e6445e823\": container with ID starting with f0caf5b618fe2940fda5d0b03d2869221c6ce498c2fd4128e9ab105e6445e823 not found: ID does not exist"
Jul 10 03:42:51.406908 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:51.406887    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"]
Jul 10 03:42:51.411246 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:51.411227    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5gbt7"]
Jul 10 03:42:53.125544 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:42:53.125518    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38c05155-aa6c-4454-847c-a12932394184" path="/var/lib/kubelet/pods/38c05155-aa6c-4454-847c-a12932394184/volumes"
Jul 10 03:43:06.767675 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.765513    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"]
Jul 10 03:43:06.767675 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.766257    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38c05155-aa6c-4454-847c-a12932394184" containerName="manager"
Jul 10 03:43:06.767675 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.766279    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="38c05155-aa6c-4454-847c-a12932394184" containerName="manager"
Jul 10 03:43:06.767675 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.766501    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="38c05155-aa6c-4454-847c-a12932394184" containerName="manager"
Jul 10 03:43:06.772590 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.772564    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.775004 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.774977    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-v4xhs\""
Jul 10 03:43:06.785020 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.784994    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"]
Jul 10 03:43:06.798134 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.798103    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.798264 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.798161    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.798264 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.798221    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.798264 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.798257    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.798439 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.798284    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.798439 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.798304    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.798439 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.798417    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwxv2\" (UniqueName: \"kubernetes.io/projected/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-kube-api-access-mwxv2\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.798594 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.798494    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.798594 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.798546    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.899623 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.899570    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.899809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.899659    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.899809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.899699    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.899809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.899735    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.899809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.899758    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.899809 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.899791    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.900093 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.899819    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.900093 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.899847    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.900093 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.899877    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwxv2\" (UniqueName: \"kubernetes.io/projected/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-kube-api-access-mwxv2\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.900221 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.900164    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.900323 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.900303    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.900413 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.900388    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.900500 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.900483    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.900580 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.900565    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.902511 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.902488    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.902632 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.902598    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.908334 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.908310    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:06.908475 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:06.908435    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwxv2\" (UniqueName: \"kubernetes.io/projected/736e2dc2-32c4-4d9a-ba85-c9122ce188a0-kube-api-access-mwxv2\") pod \"maas-default-gateway-openshift-default-56654dcf7-d5244\" (UID: \"736e2dc2-32c4-4d9a-ba85-c9122ce188a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:07.082720 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:07.082607    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:07.219776 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:07.219752    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"]
Jul 10 03:43:07.221690 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:43:07.221661    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod736e2dc2_32c4_4d9a_ba85_c9122ce188a0.slice/crio-baa3d153b12a9d9dfa356ec44d362ea5e05f0fab17d52dc10dd54b2a58bfa04c WatchSource:0}: Error finding container baa3d153b12a9d9dfa356ec44d362ea5e05f0fab17d52dc10dd54b2a58bfa04c: Status 404 returned error can't find the container with id baa3d153b12a9d9dfa356ec44d362ea5e05f0fab17d52dc10dd54b2a58bfa04c
Jul 10 03:43:07.223778 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:07.223746    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 03:43:07.223846 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:07.223814    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 03:43:07.223881 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:07.223846    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 03:43:07.457221 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:07.457188    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244" event={"ID":"736e2dc2-32c4-4d9a-ba85-c9122ce188a0","Type":"ContainerStarted","Data":"fe76e4d2974180747620b4261dba8cd3de1b762416e7ee1178f21d8a2650f62f"}
Jul 10 03:43:07.457399 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:07.457242    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244" event={"ID":"736e2dc2-32c4-4d9a-ba85-c9122ce188a0","Type":"ContainerStarted","Data":"baa3d153b12a9d9dfa356ec44d362ea5e05f0fab17d52dc10dd54b2a58bfa04c"}
Jul 10 03:43:07.481231 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:07.481173    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244" podStartSLOduration=1.481156247 podStartE2EDuration="1.481156247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:43:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:43:07.480193838 +0000 UTC m=+622.994929153" watchObservedRunningTime="2026-07-10 03:43:07.481156247 +0000 UTC m=+622.995891549"
Jul 10 03:43:08.083234 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.083197    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:08.088116 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.088095    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:08.462313 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.462284    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:08.463305 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.463284    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:43:08.530032 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.530002    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"]
Jul 10 03:43:08.533178 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.533159    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"
Jul 10 03:43:08.536200 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.536179    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 10 03:43:08.536343 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.536327    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8p5ls\""
Jul 10 03:43:08.550724 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.550696    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"]
Jul 10 03:43:08.614599 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.614561    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/10b32e75-3699-4b7a-869f-71d3d6de4b3f-config-file\") pod \"limitador-limitador-64c94fd64f-mnwkh\" (UID: \"10b32e75-3699-4b7a-869f-71d3d6de4b3f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"
Jul 10 03:43:08.614766 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.614714    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc899\" (UniqueName: \"kubernetes.io/projected/10b32e75-3699-4b7a-869f-71d3d6de4b3f-kube-api-access-dc899\") pod \"limitador-limitador-64c94fd64f-mnwkh\" (UID: \"10b32e75-3699-4b7a-869f-71d3d6de4b3f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"
Jul 10 03:43:08.653676 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.653645    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"]
Jul 10 03:43:08.715448 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.715365    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/10b32e75-3699-4b7a-869f-71d3d6de4b3f-config-file\") pod \"limitador-limitador-64c94fd64f-mnwkh\" (UID: \"10b32e75-3699-4b7a-869f-71d3d6de4b3f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"
Jul 10 03:43:08.715615 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.715536    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc899\" (UniqueName: \"kubernetes.io/projected/10b32e75-3699-4b7a-869f-71d3d6de4b3f-kube-api-access-dc899\") pod \"limitador-limitador-64c94fd64f-mnwkh\" (UID: \"10b32e75-3699-4b7a-869f-71d3d6de4b3f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"
Jul 10 03:43:08.716063 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.716035    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/10b32e75-3699-4b7a-869f-71d3d6de4b3f-config-file\") pod \"limitador-limitador-64c94fd64f-mnwkh\" (UID: \"10b32e75-3699-4b7a-869f-71d3d6de4b3f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"
Jul 10 03:43:08.724194 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.724169    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc899\" (UniqueName: \"kubernetes.io/projected/10b32e75-3699-4b7a-869f-71d3d6de4b3f-kube-api-access-dc899\") pod \"limitador-limitador-64c94fd64f-mnwkh\" (UID: \"10b32e75-3699-4b7a-869f-71d3d6de4b3f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"
Jul 10 03:43:08.844392 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.844360    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"
Jul 10 03:43:08.981533 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:08.981434    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"]
Jul 10 03:43:08.984221 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:43:08.984193    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b32e75_3699_4b7a_869f_71d3d6de4b3f.slice/crio-fcbfeea6e630a6cd4753287734f87dab7a777c85bf7a822b1eb60f265cd2c8fd WatchSource:0}: Error finding container fcbfeea6e630a6cd4753287734f87dab7a777c85bf7a822b1eb60f265cd2c8fd: Status 404 returned error can't find the container with id fcbfeea6e630a6cd4753287734f87dab7a777c85bf7a822b1eb60f265cd2c8fd
Jul 10 03:43:09.473269 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:09.473232    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh" event={"ID":"10b32e75-3699-4b7a-869f-71d3d6de4b3f","Type":"ContainerStarted","Data":"fcbfeea6e630a6cd4753287734f87dab7a777c85bf7a822b1eb60f265cd2c8fd"}
Jul 10 03:43:12.490053 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:12.490018    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh" event={"ID":"10b32e75-3699-4b7a-869f-71d3d6de4b3f","Type":"ContainerStarted","Data":"099db56b7c913a954209575939c717eeddf41fbe4bc42e9e47d82b6e6867911d"}
Jul 10 03:43:12.490444 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:12.490142    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"
Jul 10 03:43:12.506634 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:12.506576    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh" podStartSLOduration=1.83745857 podStartE2EDuration="4.506560036s" totalImagesPullingTime="2.669101466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:43:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:43:12.506509185 +0000 UTC m=+628.021244488" watchObservedRunningTime="2026-07-10 03:43:12.506560036 +0000 UTC m=+628.021295389"
Jul 10 03:43:23.475715 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:23.475681    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"]
Jul 10 03:43:23.476215 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:23.475872    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh" podUID="10b32e75-3699-4b7a-869f-71d3d6de4b3f" containerName="limitador" containerID="cri-o://099db56b7c913a954209575939c717eeddf41fbe4bc42e9e47d82b6e6867911d" gracePeriod=30
Jul 10 03:43:23.476853 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:23.476689    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"
Jul 10 03:43:24.023544 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.023510    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"
Jul 10 03:43:24.153953 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.153869    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/10b32e75-3699-4b7a-869f-71d3d6de4b3f-config-file\") pod \"10b32e75-3699-4b7a-869f-71d3d6de4b3f\" (UID: \"10b32e75-3699-4b7a-869f-71d3d6de4b3f\") "
Jul 10 03:43:24.154096 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.154007    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dc899\" (UniqueName: \"kubernetes.io/projected/10b32e75-3699-4b7a-869f-71d3d6de4b3f-kube-api-access-dc899\") pod \"10b32e75-3699-4b7a-869f-71d3d6de4b3f\" (UID: \"10b32e75-3699-4b7a-869f-71d3d6de4b3f\") "
Jul 10 03:43:24.154290 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.154268    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10b32e75-3699-4b7a-869f-71d3d6de4b3f-config-file" (OuterVolumeSpecName: "config-file") pod "10b32e75-3699-4b7a-869f-71d3d6de4b3f" (UID: "10b32e75-3699-4b7a-869f-71d3d6de4b3f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 03:43:24.156054 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.156034    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10b32e75-3699-4b7a-869f-71d3d6de4b3f-kube-api-access-dc899" (OuterVolumeSpecName: "kube-api-access-dc899") pod "10b32e75-3699-4b7a-869f-71d3d6de4b3f" (UID: "10b32e75-3699-4b7a-869f-71d3d6de4b3f"). InnerVolumeSpecName "kube-api-access-dc899". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:43:24.254645 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.254605    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dc899\" (UniqueName: \"kubernetes.io/projected/10b32e75-3699-4b7a-869f-71d3d6de4b3f-kube-api-access-dc899\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:43:24.254645 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.254636    2571 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/10b32e75-3699-4b7a-869f-71d3d6de4b3f-config-file\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:43:24.543935 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.543901    2571 generic.go:356] "Generic (PLEG): container finished" podID="10b32e75-3699-4b7a-869f-71d3d6de4b3f" containerID="099db56b7c913a954209575939c717eeddf41fbe4bc42e9e47d82b6e6867911d" exitCode=0
Jul 10 03:43:24.544327 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.543943    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"
Jul 10 03:43:24.544327 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.543979    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh" event={"ID":"10b32e75-3699-4b7a-869f-71d3d6de4b3f","Type":"ContainerDied","Data":"099db56b7c913a954209575939c717eeddf41fbe4bc42e9e47d82b6e6867911d"}
Jul 10 03:43:24.544327 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.544031    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mnwkh" event={"ID":"10b32e75-3699-4b7a-869f-71d3d6de4b3f","Type":"ContainerDied","Data":"fcbfeea6e630a6cd4753287734f87dab7a777c85bf7a822b1eb60f265cd2c8fd"}
Jul 10 03:43:24.544327 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.544053    2571 scope.go:117] "RemoveContainer" containerID="099db56b7c913a954209575939c717eeddf41fbe4bc42e9e47d82b6e6867911d"
Jul 10 03:43:24.552360 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.552337    2571 scope.go:117] "RemoveContainer" containerID="099db56b7c913a954209575939c717eeddf41fbe4bc42e9e47d82b6e6867911d"
Jul 10 03:43:24.552649 ip-10-0-143-142 kubenswrapper[2571]: E0710 03:43:24.552626    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"099db56b7c913a954209575939c717eeddf41fbe4bc42e9e47d82b6e6867911d\": container with ID starting with 099db56b7c913a954209575939c717eeddf41fbe4bc42e9e47d82b6e6867911d not found: ID does not exist" containerID="099db56b7c913a954209575939c717eeddf41fbe4bc42e9e47d82b6e6867911d"
Jul 10 03:43:24.552747 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.552657    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"099db56b7c913a954209575939c717eeddf41fbe4bc42e9e47d82b6e6867911d"} err="failed to get container status \"099db56b7c913a954209575939c717eeddf41fbe4bc42e9e47d82b6e6867911d\": rpc error: code = NotFound desc = could not find container \"099db56b7c913a954209575939c717eeddf41fbe4bc42e9e47d82b6e6867911d\": container with ID starting with 099db56b7c913a954209575939c717eeddf41fbe4bc42e9e47d82b6e6867911d not found: ID does not exist"
Jul 10 03:43:24.565222 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.565196    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"]
Jul 10 03:43:24.569397 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:24.569376    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mnwkh"]
Jul 10 03:43:25.125485 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.125439    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10b32e75-3699-4b7a-869f-71d3d6de4b3f" path="/var/lib/kubelet/pods/10b32e75-3699-4b7a-869f-71d3d6de4b3f/volumes"
Jul 10 03:43:25.176387 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.176357    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4s256"]
Jul 10 03:43:25.176692 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.176680    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10b32e75-3699-4b7a-869f-71d3d6de4b3f" containerName="limitador"
Jul 10 03:43:25.176744 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.176694    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="10b32e75-3699-4b7a-869f-71d3d6de4b3f" containerName="limitador"
Jul 10 03:43:25.176777 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.176759    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="10b32e75-3699-4b7a-869f-71d3d6de4b3f" containerName="limitador"
Jul 10 03:43:25.179767 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.179751    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4s256"
Jul 10 03:43:25.182703 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.182684    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 03:43:25.183625 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.183610    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 03:43:25.183710 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.183625    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 10 03:43:25.183710 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.183656    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-gbz6q\""
Jul 10 03:43:25.195232 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.195206    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4s256"]
Jul 10 03:43:25.263614 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.263565    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/eceb4843-a35d-4eaf-aa03-2459830a6811-data\") pod \"postgres-69fb4594cf-4s256\" (UID: \"eceb4843-a35d-4eaf-aa03-2459830a6811\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4s256"
Jul 10 03:43:25.263614 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.263618    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsgnj\" (UniqueName: \"kubernetes.io/projected/eceb4843-a35d-4eaf-aa03-2459830a6811-kube-api-access-nsgnj\") pod \"postgres-69fb4594cf-4s256\" (UID: \"eceb4843-a35d-4eaf-aa03-2459830a6811\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4s256"
Jul 10 03:43:25.365171 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.365135    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/eceb4843-a35d-4eaf-aa03-2459830a6811-data\") pod \"postgres-69fb4594cf-4s256\" (UID: \"eceb4843-a35d-4eaf-aa03-2459830a6811\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4s256"
Jul 10 03:43:25.365171 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.365170    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsgnj\" (UniqueName: \"kubernetes.io/projected/eceb4843-a35d-4eaf-aa03-2459830a6811-kube-api-access-nsgnj\") pod \"postgres-69fb4594cf-4s256\" (UID: \"eceb4843-a35d-4eaf-aa03-2459830a6811\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4s256"
Jul 10 03:43:25.365560 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.365541    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/eceb4843-a35d-4eaf-aa03-2459830a6811-data\") pod \"postgres-69fb4594cf-4s256\" (UID: \"eceb4843-a35d-4eaf-aa03-2459830a6811\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4s256"
Jul 10 03:43:25.373100 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.373075    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsgnj\" (UniqueName: \"kubernetes.io/projected/eceb4843-a35d-4eaf-aa03-2459830a6811-kube-api-access-nsgnj\") pod \"postgres-69fb4594cf-4s256\" (UID: \"eceb4843-a35d-4eaf-aa03-2459830a6811\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4s256"
Jul 10 03:43:25.489638 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.489607    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4s256"
Jul 10 03:43:25.643878 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:25.643849    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4s256"]
Jul 10 03:43:25.648152 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:43:25.648127    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeceb4843_a35d_4eaf_aa03_2459830a6811.slice/crio-632511ec6416f1e412534d6c493f359c464402439801a2ebfff797dd9c06e156 WatchSource:0}: Error finding container 632511ec6416f1e412534d6c493f359c464402439801a2ebfff797dd9c06e156: Status 404 returned error can't find the container with id 632511ec6416f1e412534d6c493f359c464402439801a2ebfff797dd9c06e156
Jul 10 03:43:26.569423 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:26.569381    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4s256" event={"ID":"eceb4843-a35d-4eaf-aa03-2459830a6811","Type":"ContainerStarted","Data":"632511ec6416f1e412534d6c493f359c464402439801a2ebfff797dd9c06e156"}
Jul 10 03:43:30.595181 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:30.595145    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4s256" event={"ID":"eceb4843-a35d-4eaf-aa03-2459830a6811","Type":"ContainerStarted","Data":"1956a87b2c71a740913656097e4258026c3e99477ccc942391e33b0c8a2b89f3"}
Jul 10 03:43:30.595683 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:30.595261    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4s256"
Jul 10 03:43:30.611801 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:30.611752    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4s256" podStartSLOduration=0.892992743 podStartE2EDuration="5.611738962s" totalImagesPullingTime="4.718746219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:43:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:43:30.61037459 +0000 UTC m=+646.125109875" watchObservedRunningTime="2026-07-10 03:43:30.611738962 +0000 UTC m=+646.126474278"
Jul 10 03:43:36.628707 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:43:36.628675    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4s256"
Jul 10 03:44:18.361178 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.361142    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"]
Jul 10 03:44:18.364320 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.364298    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"
Jul 10 03:44:18.368102 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.368077    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6g9gw\""
Jul 10 03:44:18.368102 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.368098    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 10 03:44:18.378689 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.378665    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"]
Jul 10 03:44:18.433273 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.433235    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4ae5589-365b-4565-b0a4-ba128bc8297e-maas-api-tls\") pod \"maas-api-6dbbf545f9-mdtmj\" (UID: \"b4ae5589-365b-4565-b0a4-ba128bc8297e\") " pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"
Jul 10 03:44:18.433432 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.433338    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4ae5589-365b-4565-b0a4-ba128bc8297e-openshift-service-ca\") pod \"maas-api-6dbbf545f9-mdtmj\" (UID: \"b4ae5589-365b-4565-b0a4-ba128bc8297e\") " pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"
Jul 10 03:44:18.433432 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.433380    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swspj\" (UniqueName: \"kubernetes.io/projected/b4ae5589-365b-4565-b0a4-ba128bc8297e-kube-api-access-swspj\") pod \"maas-api-6dbbf545f9-mdtmj\" (UID: \"b4ae5589-365b-4565-b0a4-ba128bc8297e\") " pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"
Jul 10 03:44:18.534604 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.534568    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4ae5589-365b-4565-b0a4-ba128bc8297e-openshift-service-ca\") pod \"maas-api-6dbbf545f9-mdtmj\" (UID: \"b4ae5589-365b-4565-b0a4-ba128bc8297e\") " pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"
Jul 10 03:44:18.534785 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.534615    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swspj\" (UniqueName: \"kubernetes.io/projected/b4ae5589-365b-4565-b0a4-ba128bc8297e-kube-api-access-swspj\") pod \"maas-api-6dbbf545f9-mdtmj\" (UID: \"b4ae5589-365b-4565-b0a4-ba128bc8297e\") " pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"
Jul 10 03:44:18.534785 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.534653    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4ae5589-365b-4565-b0a4-ba128bc8297e-maas-api-tls\") pod \"maas-api-6dbbf545f9-mdtmj\" (UID: \"b4ae5589-365b-4565-b0a4-ba128bc8297e\") " pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"
Jul 10 03:44:18.535261 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.535237    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4ae5589-365b-4565-b0a4-ba128bc8297e-openshift-service-ca\") pod \"maas-api-6dbbf545f9-mdtmj\" (UID: \"b4ae5589-365b-4565-b0a4-ba128bc8297e\") " pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"
Jul 10 03:44:18.536990 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.536969    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4ae5589-365b-4565-b0a4-ba128bc8297e-maas-api-tls\") pod \"maas-api-6dbbf545f9-mdtmj\" (UID: \"b4ae5589-365b-4565-b0a4-ba128bc8297e\") " pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"
Jul 10 03:44:18.543651 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.543628    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swspj\" (UniqueName: \"kubernetes.io/projected/b4ae5589-365b-4565-b0a4-ba128bc8297e-kube-api-access-swspj\") pod \"maas-api-6dbbf545f9-mdtmj\" (UID: \"b4ae5589-365b-4565-b0a4-ba128bc8297e\") " pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"
Jul 10 03:44:18.674503 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.674455    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"
Jul 10 03:44:18.847582 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:18.847554    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"]
Jul 10 03:44:18.849122 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:44:18.849096    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ae5589_365b_4565_b0a4_ba128bc8297e.slice/crio-14ed047851c20c06aa09ea071f43b38ff40ab3ed6a533e466422d3ed522de649 WatchSource:0}: Error finding container 14ed047851c20c06aa09ea071f43b38ff40ab3ed6a533e466422d3ed522de649: Status 404 returned error can't find the container with id 14ed047851c20c06aa09ea071f43b38ff40ab3ed6a533e466422d3ed522de649
Jul 10 03:44:19.839484 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:19.839426    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj" event={"ID":"b4ae5589-365b-4565-b0a4-ba128bc8297e","Type":"ContainerStarted","Data":"14ed047851c20c06aa09ea071f43b38ff40ab3ed6a533e466422d3ed522de649"}
Jul 10 03:44:21.856669 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:21.856636    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj" event={"ID":"b4ae5589-365b-4565-b0a4-ba128bc8297e","Type":"ContainerStarted","Data":"ef2994e851ccf1cd3459927adf7100e8151bfeb34755cf132841f3bd117ca50f"}
Jul 10 03:44:21.857130 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:21.856812    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"
Jul 10 03:44:21.873259 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:21.873214    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj" podStartSLOduration=1.030026161 podStartE2EDuration="3.87319627s" totalImagesPullingTime="2.843170109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:44:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:44:21.872310151 +0000 UTC m=+697.387045452" watchObservedRunningTime="2026-07-10 03:44:21.87319627 +0000 UTC m=+697.387931571"
Jul 10 03:44:26.888281 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:26.888184    2571 generic.go:356] "Generic (PLEG): container finished" podID="736e2dc2-32c4-4d9a-ba85-c9122ce188a0" containerID="fe76e4d2974180747620b4261dba8cd3de1b762416e7ee1178f21d8a2650f62f" exitCode=0
Jul 10 03:44:26.888281 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:26.888265    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244" event={"ID":"736e2dc2-32c4-4d9a-ba85-c9122ce188a0","Type":"ContainerDied","Data":"fe76e4d2974180747620b4261dba8cd3de1b762416e7ee1178f21d8a2650f62f"}
Jul 10 03:44:26.888794 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:26.888777    2571 scope.go:117] "RemoveContainer" containerID="fe76e4d2974180747620b4261dba8cd3de1b762416e7ee1178f21d8a2650f62f"
Jul 10 03:44:26.889739 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:26.889706    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 03:44:26.889853 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:26.889784    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 03:44:26.889853 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:26.889834    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 03:44:27.873248 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:27.873215    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6dbbf545f9-mdtmj"
Jul 10 03:44:27.894750 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:27.894717    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244" event={"ID":"736e2dc2-32c4-4d9a-ba85-c9122ce188a0","Type":"ContainerStarted","Data":"d173457829a6a57d830564c545856b0827fdcceb8c5951685efeec23e25ca8ce"}
Jul 10 03:44:28.083199 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:28.083159    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:44:28.083396 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:28.083375    2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-d5244 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 10 03:44:28.083441 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:28.083424    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244" podUID="736e2dc2-32c4-4d9a-ba85-c9122ce188a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 10 03:44:29.083028 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:29.082993    2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-d5244 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 10 03:44:29.083411 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:29.083060    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244" podUID="736e2dc2-32c4-4d9a-ba85-c9122ce188a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 10 03:44:30.083300 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:30.083263    2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-d5244 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 10 03:44:30.083756 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:30.083315    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244" podUID="736e2dc2-32c4-4d9a-ba85-c9122ce188a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 10 03:44:31.083595 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:31.083556    2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-d5244 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 10 03:44:31.084015 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:31.083620    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244" podUID="736e2dc2-32c4-4d9a-ba85-c9122ce188a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 10 03:44:32.088494 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:32.088445    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:44:32.088892 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:32.088739    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:44:32.089512 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:32.089492    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-d5244"
Jul 10 03:44:39.866318 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:39.866249    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"]
Jul 10 03:44:39.875358 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:39.875333    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:39.878455 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:39.878427    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 10 03:44:39.878606 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:39.878427    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vxxnm\""
Jul 10 03:44:39.878606 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:39.878560    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 10 03:44:39.878719 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:39.878703    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 10 03:44:39.881635 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:39.881612    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"]
Jul 10 03:44:39.925642 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:39.925605    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd3f1056-33b8-40ec-8c38-cbc930677fa4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:39.925642 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:39.925642    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5kbz\" (UniqueName: \"kubernetes.io/projected/fd3f1056-33b8-40ec-8c38-cbc930677fa4-kube-api-access-l5kbz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:39.925860 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:39.925679    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd3f1056-33b8-40ec-8c38-cbc930677fa4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:39.925860 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:39.925724    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd3f1056-33b8-40ec-8c38-cbc930677fa4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:39.925860 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:39.925745    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd3f1056-33b8-40ec-8c38-cbc930677fa4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:39.925860 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:39.925800    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd3f1056-33b8-40ec-8c38-cbc930677fa4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.026309 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.026276    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd3f1056-33b8-40ec-8c38-cbc930677fa4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.026309 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.026316    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5kbz\" (UniqueName: \"kubernetes.io/projected/fd3f1056-33b8-40ec-8c38-cbc930677fa4-kube-api-access-l5kbz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.026632 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.026372    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd3f1056-33b8-40ec-8c38-cbc930677fa4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.026632 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.026426    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd3f1056-33b8-40ec-8c38-cbc930677fa4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.026632 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.026577    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd3f1056-33b8-40ec-8c38-cbc930677fa4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.026792 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.026673    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd3f1056-33b8-40ec-8c38-cbc930677fa4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.026792 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.026773    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd3f1056-33b8-40ec-8c38-cbc930677fa4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.027005 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.026978    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd3f1056-33b8-40ec-8c38-cbc930677fa4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.027005 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.026994    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd3f1056-33b8-40ec-8c38-cbc930677fa4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.029267 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.029243    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd3f1056-33b8-40ec-8c38-cbc930677fa4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.029503 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.029480    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd3f1056-33b8-40ec-8c38-cbc930677fa4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.033060 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.033036    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5kbz\" (UniqueName: \"kubernetes.io/projected/fd3f1056-33b8-40ec-8c38-cbc930677fa4-kube-api-access-l5kbz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hf6fh\" (UID: \"fd3f1056-33b8-40ec-8c38-cbc930677fa4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.188703 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.188665    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:40.326654 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.326625    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"]
Jul 10 03:44:40.328418 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:44:40.328385    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd3f1056_33b8_40ec_8c38_cbc930677fa4.slice/crio-c2fce52c6be6166152eded3ae0c6f99037dc52beb1e26de7fce73453fb5494a6 WatchSource:0}: Error finding container c2fce52c6be6166152eded3ae0c6f99037dc52beb1e26de7fce73453fb5494a6: Status 404 returned error can't find the container with id c2fce52c6be6166152eded3ae0c6f99037dc52beb1e26de7fce73453fb5494a6
Jul 10 03:44:40.956341 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:40.956299    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh" event={"ID":"fd3f1056-33b8-40ec-8c38-cbc930677fa4","Type":"ContainerStarted","Data":"c2fce52c6be6166152eded3ae0c6f99037dc52beb1e26de7fce73453fb5494a6"}
Jul 10 03:44:41.963829 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:41.963795    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh" event={"ID":"fd3f1056-33b8-40ec-8c38-cbc930677fa4","Type":"ContainerStarted","Data":"15bb5dccef91d9b37704d877ab485e26e8bf800bd23445b9709152430c1c5987"}
Jul 10 03:44:50.189556 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:50.189513    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:50.190193 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:50.189593    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:50.202008 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:50.201975    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:50.218694 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:50.218647    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh" podStartSLOduration=10.258487544 podStartE2EDuration="11.218632204s" totalImagesPullingTime="960.14466ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:44:41.979257125 +0000 UTC m=+717.493992425" watchObservedRunningTime="2026-07-10 03:44:50.218632204 +0000 UTC m=+725.733367504"
Jul 10 03:44:51.015700 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:51.015672    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hf6fh"
Jul 10 03:44:58.776901 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.776865    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"]
Jul 10 03:44:58.779419 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.779400    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.781536 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.781515    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 10 03:44:58.789662 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.789633    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"]
Jul 10 03:44:58.877117 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.877087    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f69d2bf-5584-41c3-a513-65b127f066c6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.877301 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.877128    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm52d\" (UniqueName: \"kubernetes.io/projected/5f69d2bf-5584-41c3-a513-65b127f066c6-kube-api-access-mm52d\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.877301 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.877150    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f69d2bf-5584-41c3-a513-65b127f066c6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.877301 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.877174    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f69d2bf-5584-41c3-a513-65b127f066c6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.877301 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.877200    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f69d2bf-5584-41c3-a513-65b127f066c6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.877301 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.877236    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f69d2bf-5584-41c3-a513-65b127f066c6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.978215 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.978169    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f69d2bf-5584-41c3-a513-65b127f066c6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.978401 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.978230    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f69d2bf-5584-41c3-a513-65b127f066c6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.978401 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.978271    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm52d\" (UniqueName: \"kubernetes.io/projected/5f69d2bf-5584-41c3-a513-65b127f066c6-kube-api-access-mm52d\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.978401 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.978300    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f69d2bf-5584-41c3-a513-65b127f066c6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.978615 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.978494    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f69d2bf-5584-41c3-a513-65b127f066c6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.978615 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.978552    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f69d2bf-5584-41c3-a513-65b127f066c6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.978722 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.978698    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f69d2bf-5584-41c3-a513-65b127f066c6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.978863 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.978843    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f69d2bf-5584-41c3-a513-65b127f066c6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.978863 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.978858    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f69d2bf-5584-41c3-a513-65b127f066c6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.980494 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.980456    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f69d2bf-5584-41c3-a513-65b127f066c6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.980789 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.980771    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f69d2bf-5584-41c3-a513-65b127f066c6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:58.986156 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:58.986137    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm52d\" (UniqueName: \"kubernetes.io/projected/5f69d2bf-5584-41c3-a513-65b127f066c6-kube-api-access-mm52d\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zkjfs\" (UID: \"5f69d2bf-5584-41c3-a513-65b127f066c6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:59.090737 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:59.090649    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:44:59.218605 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:44:59.218566    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"]
Jul 10 03:44:59.222733 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:44:59.222703    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f69d2bf_5584_41c3_a513_65b127f066c6.slice/crio-e4aaa8c9fd811cc024bec5a2b4e346b5020756c6a187ad27abf5af15d72e2083 WatchSource:0}: Error finding container e4aaa8c9fd811cc024bec5a2b4e346b5020756c6a187ad27abf5af15d72e2083: Status 404 returned error can't find the container with id e4aaa8c9fd811cc024bec5a2b4e346b5020756c6a187ad27abf5af15d72e2083
Jul 10 03:45:00.048092 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:00.048056    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs" event={"ID":"5f69d2bf-5584-41c3-a513-65b127f066c6","Type":"ContainerStarted","Data":"385212ede3866f098383ec4242e0180ae8b0826ab7c3e15603e76e90f7cdfd8b"}
Jul 10 03:45:00.048571 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:00.048147    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs" event={"ID":"5f69d2bf-5584-41c3-a513-65b127f066c6","Type":"ContainerStarted","Data":"e4aaa8c9fd811cc024bec5a2b4e346b5020756c6a187ad27abf5af15d72e2083"}
Jul 10 03:45:00.064626 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:00.064576    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs" podStartSLOduration=1.9052474400000001 podStartE2EDuration="2.064558407s" totalImagesPullingTime="159.310967ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:45:00.063941324 +0000 UTC m=+735.578676628" watchObservedRunningTime="2026-07-10 03:45:00.064558407 +0000 UTC m=+735.579293709"
Jul 10 03:45:09.091046 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:09.091012    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:45:09.091580 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:09.091085    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:45:09.103230 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:09.103208    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:45:10.106917 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:10.106885    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zkjfs"
Jul 10 03:45:42.257443 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.257396    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"]
Jul 10 03:45:42.261422 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.261394    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.263497 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.263449    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 10 03:45:42.271898 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.271874    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"]
Jul 10 03:45:42.366933 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.366891    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c625d684-5d06-49e5-a901-356978c2b090-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.367110 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.366971    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c625d684-5d06-49e5-a901-356978c2b090-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.367110 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.366993    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c625d684-5d06-49e5-a901-356978c2b090-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.367110 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.367045    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c625d684-5d06-49e5-a901-356978c2b090-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.367110 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.367078    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c625d684-5d06-49e5-a901-356978c2b090-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.367110 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.367095    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n29dm\" (UniqueName: \"kubernetes.io/projected/c625d684-5d06-49e5-a901-356978c2b090-kube-api-access-n29dm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.467690 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.467644    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c625d684-5d06-49e5-a901-356978c2b090-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.467690 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.467690    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n29dm\" (UniqueName: \"kubernetes.io/projected/c625d684-5d06-49e5-a901-356978c2b090-kube-api-access-n29dm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.467952 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.467742    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c625d684-5d06-49e5-a901-356978c2b090-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.467952 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.467816    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c625d684-5d06-49e5-a901-356978c2b090-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.467952 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.467849    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c625d684-5d06-49e5-a901-356978c2b090-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.467952 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.467891    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c625d684-5d06-49e5-a901-356978c2b090-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.468132 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.468053    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c625d684-5d06-49e5-a901-356978c2b090-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.468312 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.468290    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c625d684-5d06-49e5-a901-356978c2b090-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.468378 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.468298    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c625d684-5d06-49e5-a901-356978c2b090-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.470002 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.469977    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c625d684-5d06-49e5-a901-356978c2b090-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.470243 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.470226    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c625d684-5d06-49e5-a901-356978c2b090-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.474794 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.474774    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n29dm\" (UniqueName: \"kubernetes.io/projected/c625d684-5d06-49e5-a901-356978c2b090-kube-api-access-n29dm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-md567\" (UID: \"c625d684-5d06-49e5-a901-356978c2b090\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.572895 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.572810    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:42.710110 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.710082    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"]
Jul 10 03:45:42.712819 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:45:42.712785    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc625d684_5d06_49e5_a901_356978c2b090.slice/crio-9e10ee0c0ddbd5a3707263dbaab2378db1e33d105eab06e0351f8023705aa689 WatchSource:0}: Error finding container 9e10ee0c0ddbd5a3707263dbaab2378db1e33d105eab06e0351f8023705aa689: Status 404 returned error can't find the container with id 9e10ee0c0ddbd5a3707263dbaab2378db1e33d105eab06e0351f8023705aa689
Jul 10 03:45:42.714090 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:42.714074    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 03:45:43.248075 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:43.248034    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567" event={"ID":"c625d684-5d06-49e5-a901-356978c2b090","Type":"ContainerStarted","Data":"376470d154b2a417071ebddb5a36f7dbce8f7888796237b20f1b8bd6e182f6b3"}
Jul 10 03:45:43.248075 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:43.248077    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567" event={"ID":"c625d684-5d06-49e5-a901-356978c2b090","Type":"ContainerStarted","Data":"9e10ee0c0ddbd5a3707263dbaab2378db1e33d105eab06e0351f8023705aa689"}
Jul 10 03:45:43.265707 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:43.265646    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567" podStartSLOduration=1.100198454 podStartE2EDuration="1.26562568s" totalImagesPullingTime="165.427226ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:45:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:45:43.263621942 +0000 UTC m=+778.778357246" watchObservedRunningTime="2026-07-10 03:45:43.26562568 +0000 UTC m=+778.780360982"
Jul 10 03:45:52.573966 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:52.573921    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:52.574513 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:52.573998    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:52.586290 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:52.586268    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:45:53.304436 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:45:53.304410    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-md567"
Jul 10 03:52:20.033080 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:20.033005    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"]
Jul 10 03:52:20.033619 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:20.033234    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf" podUID="e03b04f6-7657-4483-9ee8-eefc2c92749b" containerName="manager" containerID="cri-o://46fc8c220efd08b1c25e21676f95f639164aa41c4087af994f7b11f9078d7cca" gracePeriod=10
Jul 10 03:52:23.084000 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:23.083973    2571 generic.go:356] "Generic (PLEG): container finished" podID="e03b04f6-7657-4483-9ee8-eefc2c92749b" containerID="46fc8c220efd08b1c25e21676f95f639164aa41c4087af994f7b11f9078d7cca" exitCode=0
Jul 10 03:52:23.084330 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:23.084031    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf" event={"ID":"e03b04f6-7657-4483-9ee8-eefc2c92749b","Type":"ContainerDied","Data":"46fc8c220efd08b1c25e21676f95f639164aa41c4087af994f7b11f9078d7cca"}
Jul 10 03:52:23.101520 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:23.101492    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"
Jul 10 03:52:23.260935 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:23.260846    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wtz2\" (UniqueName: \"kubernetes.io/projected/e03b04f6-7657-4483-9ee8-eefc2c92749b-kube-api-access-8wtz2\") pod \"e03b04f6-7657-4483-9ee8-eefc2c92749b\" (UID: \"e03b04f6-7657-4483-9ee8-eefc2c92749b\") "
Jul 10 03:52:23.260935 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:23.260913    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e03b04f6-7657-4483-9ee8-eefc2c92749b-extensions-socket-volume\") pod \"e03b04f6-7657-4483-9ee8-eefc2c92749b\" (UID: \"e03b04f6-7657-4483-9ee8-eefc2c92749b\") "
Jul 10 03:52:23.261316 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:23.261290    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03b04f6-7657-4483-9ee8-eefc2c92749b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e03b04f6-7657-4483-9ee8-eefc2c92749b" (UID: "e03b04f6-7657-4483-9ee8-eefc2c92749b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 03:52:23.262937 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:23.262914    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e03b04f6-7657-4483-9ee8-eefc2c92749b-kube-api-access-8wtz2" (OuterVolumeSpecName: "kube-api-access-8wtz2") pod "e03b04f6-7657-4483-9ee8-eefc2c92749b" (UID: "e03b04f6-7657-4483-9ee8-eefc2c92749b"). InnerVolumeSpecName "kube-api-access-8wtz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 03:52:23.361869 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:23.361828    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wtz2\" (UniqueName: \"kubernetes.io/projected/e03b04f6-7657-4483-9ee8-eefc2c92749b-kube-api-access-8wtz2\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:52:23.361869 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:23.361860    2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e03b04f6-7657-4483-9ee8-eefc2c92749b-extensions-socket-volume\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 03:52:24.089325 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:24.089294    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"
Jul 10 03:52:24.089325 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:24.089304    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf" event={"ID":"e03b04f6-7657-4483-9ee8-eefc2c92749b","Type":"ContainerDied","Data":"4d4d8d525fd1b3f57d4e0f8c8f0cfb3edfe263e17f2b3b649492e34706a15313"}
Jul 10 03:52:24.089805 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:24.089358    2571 scope.go:117] "RemoveContainer" containerID="46fc8c220efd08b1c25e21676f95f639164aa41c4087af994f7b11f9078d7cca"
Jul 10 03:52:24.108739 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:24.108711    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"]
Jul 10 03:52:24.112729 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:24.112703    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9xdf"]
Jul 10 03:52:25.126760 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:52:25.126733    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e03b04f6-7657-4483-9ee8-eefc2c92749b" path="/var/lib/kubelet/pods/e03b04f6-7657-4483-9ee8-eefc2c92749b/volumes"
Jul 10 03:53:21.231072 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.231040    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4"]
Jul 10 03:53:21.231598 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.231386    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e03b04f6-7657-4483-9ee8-eefc2c92749b" containerName="manager"
Jul 10 03:53:21.231598 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.231396    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e03b04f6-7657-4483-9ee8-eefc2c92749b" containerName="manager"
Jul 10 03:53:21.231598 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.231454    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e03b04f6-7657-4483-9ee8-eefc2c92749b" containerName="manager"
Jul 10 03:53:21.236485 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.236446    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4"
Jul 10 03:53:21.238792 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.238771    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vh552\""
Jul 10 03:53:21.246718 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.246695    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4"]
Jul 10 03:53:21.330747 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.330702    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a837360a-4250-446f-bc24-d71f7a422884-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-msdr4\" (UID: \"a837360a-4250-446f-bc24-d71f7a422884\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4"
Jul 10 03:53:21.330942 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.330785    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6dlc\" (UniqueName: \"kubernetes.io/projected/a837360a-4250-446f-bc24-d71f7a422884-kube-api-access-n6dlc\") pod \"kuadrant-operator-controller-manager-86878c769f-msdr4\" (UID: \"a837360a-4250-446f-bc24-d71f7a422884\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4"
Jul 10 03:53:21.431811 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.431769    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a837360a-4250-446f-bc24-d71f7a422884-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-msdr4\" (UID: \"a837360a-4250-446f-bc24-d71f7a422884\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4"
Jul 10 03:53:21.431977 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.431826    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6dlc\" (UniqueName: \"kubernetes.io/projected/a837360a-4250-446f-bc24-d71f7a422884-kube-api-access-n6dlc\") pod \"kuadrant-operator-controller-manager-86878c769f-msdr4\" (UID: \"a837360a-4250-446f-bc24-d71f7a422884\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4"
Jul 10 03:53:21.432158 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.432137    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a837360a-4250-446f-bc24-d71f7a422884-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-msdr4\" (UID: \"a837360a-4250-446f-bc24-d71f7a422884\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4"
Jul 10 03:53:21.443422 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.443391    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6dlc\" (UniqueName: \"kubernetes.io/projected/a837360a-4250-446f-bc24-d71f7a422884-kube-api-access-n6dlc\") pod \"kuadrant-operator-controller-manager-86878c769f-msdr4\" (UID: \"a837360a-4250-446f-bc24-d71f7a422884\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4"
Jul 10 03:53:21.546959 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.546877    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4"
Jul 10 03:53:21.693377 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:21.693351    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4"]
Jul 10 03:53:21.696107 ip-10-0-143-142 kubenswrapper[2571]: W0710 03:53:21.696078    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda837360a_4250_446f_bc24_d71f7a422884.slice/crio-c5004b9dc841ab5792943608a6d5134a7d4b5c1e0c91f7759e3a8f96e32daaef WatchSource:0}: Error finding container c5004b9dc841ab5792943608a6d5134a7d4b5c1e0c91f7759e3a8f96e32daaef: Status 404 returned error can't find the container with id c5004b9dc841ab5792943608a6d5134a7d4b5c1e0c91f7759e3a8f96e32daaef
Jul 10 03:53:22.357263 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:22.357211    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4" event={"ID":"a837360a-4250-446f-bc24-d71f7a422884","Type":"ContainerStarted","Data":"0bb309578aa35844f865678c0ea47cc4f971470eab4ef5b667a1e4ff8ab7c01f"}
Jul 10 03:53:22.357716 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:22.357277    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4" event={"ID":"a837360a-4250-446f-bc24-d71f7a422884","Type":"ContainerStarted","Data":"c5004b9dc841ab5792943608a6d5134a7d4b5c1e0c91f7759e3a8f96e32daaef"}
Jul 10 03:53:22.357716 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:22.357386    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4"
Jul 10 03:53:22.375960 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:22.375911    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4" podStartSLOduration=1.375896076 podStartE2EDuration="1.375896076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:53:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:53:22.373997834 +0000 UTC m=+1237.888733135" watchObservedRunningTime="2026-07-10 03:53:22.375896076 +0000 UTC m=+1237.890631376"
Jul 10 03:53:33.365274 ip-10-0-143-142 kubenswrapper[2571]: I0710 03:53:33.365191    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-msdr4"
Jul 10 04:01:09.974092 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:09.974005    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"]
Jul 10 04:01:09.977709 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:09.977685    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:09.980165 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:09.980140    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8d8a2105-openshift-default-dockercfg-pggxn\""
Jul 10 04:01:09.989144 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:09.989122    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"]
Jul 10 04:01:10.063290 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.063255    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99da311f-1d48-4128-b69a-94b50a66a807-istio-token\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.063290 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.063289    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-workload-socket\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.063526 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.063313    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-credential-socket\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.063526 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.063370    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-workload-certs\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.063526 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.063454    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-istio-data\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.063623 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.063569    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99da311f-1d48-4128-b69a-94b50a66a807-istio-podinfo\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.063623 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.063610    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99da311f-1d48-4128-b69a-94b50a66a807-istiod-ca-cert\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.063687 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.063632    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-istio-envoy\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.063687 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.063671    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j879q\" (UniqueName: \"kubernetes.io/projected/99da311f-1d48-4128-b69a-94b50a66a807-kube-api-access-j879q\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.164337 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.164299    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99da311f-1d48-4128-b69a-94b50a66a807-istio-token\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.164337 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.164334    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-workload-socket\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.164625 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.164354    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-credential-socket\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.164625 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.164370    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-workload-certs\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.164625 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.164574    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-istio-data\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.164793 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.164667    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99da311f-1d48-4128-b69a-94b50a66a807-istio-podinfo\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.164793 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.164701    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99da311f-1d48-4128-b69a-94b50a66a807-istiod-ca-cert\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.164793 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.164733    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-istio-envoy\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.164917 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.164832    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-workload-socket\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.164917 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.164792    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-credential-socket\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.164917 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.164769    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-workload-certs\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.164917 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.164884    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j879q\" (UniqueName: \"kubernetes.io/projected/99da311f-1d48-4128-b69a-94b50a66a807-kube-api-access-j879q\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.165063 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.164935    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-istio-data\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.165398 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.165376    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99da311f-1d48-4128-b69a-94b50a66a807-istiod-ca-cert\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.166984 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.166958    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-istio-envoy\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.167121 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.167104    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99da311f-1d48-4128-b69a-94b50a66a807-istio-podinfo\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.172015 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.171991    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99da311f-1d48-4128-b69a-94b50a66a807-istio-token\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.172110 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.172097    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j879q\" (UniqueName: \"kubernetes.io/projected/99da311f-1d48-4128-b69a-94b50a66a807-kube-api-access-j879q\") pod \"e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") " pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.290168 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.290082    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:10.431440 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.431415    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"]
Jul 10 04:01:10.433756 ip-10-0-143-142 kubenswrapper[2571]: W0710 04:01:10.433725    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99da311f_1d48_4128_b69a_94b50a66a807.slice/crio-901f7ba077517af0542dacc24b74cf053d26de5e8744e92b5cd2b40a43a67417 WatchSource:0}: Error finding container 901f7ba077517af0542dacc24b74cf053d26de5e8744e92b5cd2b40a43a67417: Status 404 returned error can't find the container with id 901f7ba077517af0542dacc24b74cf053d26de5e8744e92b5cd2b40a43a67417
Jul 10 04:01:10.436166 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.436116    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:01:10.436307 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.436225    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:01:10.436307 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.436274    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:01:10.532377 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.532342    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n" event={"ID":"99da311f-1d48-4128-b69a-94b50a66a807","Type":"ContainerStarted","Data":"6ff5119ed1c31b652665803f241a105eb85fcd984ac1676249f0a078f9a1f0c1"}
Jul 10 04:01:10.532579 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.532407    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n" event={"ID":"99da311f-1d48-4128-b69a-94b50a66a807","Type":"ContainerStarted","Data":"901f7ba077517af0542dacc24b74cf053d26de5e8744e92b5cd2b40a43a67417"}
Jul 10 04:01:10.552760 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:10.552655    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n" podStartSLOduration=1.552635808 podStartE2EDuration="1.552635808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:01:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:01:10.550361655 +0000 UTC m=+1706.065096960" watchObservedRunningTime="2026-07-10 04:01:10.552635808 +0000 UTC m=+1706.067371112"
Jul 10 04:01:11.291201 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:11.291167    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:11.295954 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:11.295932    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:11.537083 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:11.537049    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:11.537970 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:11.537952    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:22.588793 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:22.588758    2571 generic.go:356] "Generic (PLEG): container finished" podID="99da311f-1d48-4128-b69a-94b50a66a807" containerID="6ff5119ed1c31b652665803f241a105eb85fcd984ac1676249f0a078f9a1f0c1" exitCode=0
Jul 10 04:01:22.589212 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:22.588831    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n" event={"ID":"99da311f-1d48-4128-b69a-94b50a66a807","Type":"ContainerDied","Data":"6ff5119ed1c31b652665803f241a105eb85fcd984ac1676249f0a078f9a1f0c1"}
Jul 10 04:01:22.589257 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:22.589224    2571 scope.go:117] "RemoveContainer" containerID="6ff5119ed1c31b652665803f241a105eb85fcd984ac1676249f0a078f9a1f0c1"
Jul 10 04:01:22.590250 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:22.590223    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:01:22.590313 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:22.590289    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:01:22.590367 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:22.590328    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:01:23.598417 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:23.598383    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n" event={"ID":"99da311f-1d48-4128-b69a-94b50a66a807","Type":"ContainerStarted","Data":"af51b045818c6c7a3033e87669c4a854d75c9c6486645b7658d4ffd91cf68884"}
Jul 10 04:01:24.291024 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:24.290978    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:24.292296 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:24.292270    2571 patch_prober.go:28] interesting pod/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 10 04:01:24.292424 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:24.292320    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n" podUID="99da311f-1d48-4128-b69a-94b50a66a807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 10 04:01:25.290891 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:25.290853    2571 patch_prober.go:28] interesting pod/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 10 04:01:25.291290 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:25.290911    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n" podUID="99da311f-1d48-4128-b69a-94b50a66a807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 10 04:01:26.290716 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:26.290680    2571 patch_prober.go:28] interesting pod/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 10 04:01:26.290899 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:26.290735    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n" podUID="99da311f-1d48-4128-b69a-94b50a66a807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 10 04:01:26.537558 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:26.537515    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:27.290826 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:27.290790    2571 patch_prober.go:28] interesting pod/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 10 04:01:27.291017 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:27.290847    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n" podUID="99da311f-1d48-4128-b69a-94b50a66a807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 10 04:01:28.294279 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:28.294246    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:01:28.295379 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:01:28.295362    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:02:32.419777 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:32.419740    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"]
Jul 10 04:02:32.420374 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:32.420019    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n" podUID="99da311f-1d48-4128-b69a-94b50a66a807" containerName="istio-proxy" containerID="cri-o://af51b045818c6c7a3033e87669c4a854d75c9c6486645b7658d4ffd91cf68884" gracePeriod=30
Jul 10 04:02:37.680245 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.680183    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:02:37.790125 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.789726    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-credential-socket\") pod \"99da311f-1d48-4128-b69a-94b50a66a807\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") "
Jul 10 04:02:37.790357 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.790310    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "99da311f-1d48-4128-b69a-94b50a66a807" (UID: "99da311f-1d48-4128-b69a-94b50a66a807"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:02:37.790496 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.790449    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-istio-data\") pod \"99da311f-1d48-4128-b69a-94b50a66a807\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") "
Jul 10 04:02:37.790665 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.790650    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-workload-certs\") pod \"99da311f-1d48-4128-b69a-94b50a66a807\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") "
Jul 10 04:02:37.790827 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.790813    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99da311f-1d48-4128-b69a-94b50a66a807-istiod-ca-cert\") pod \"99da311f-1d48-4128-b69a-94b50a66a807\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") "
Jul 10 04:02:37.790938 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.790926    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-workload-socket\") pod \"99da311f-1d48-4128-b69a-94b50a66a807\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") "
Jul 10 04:02:37.791062 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.791050    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-istio-envoy\") pod \"99da311f-1d48-4128-b69a-94b50a66a807\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") "
Jul 10 04:02:37.791187 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.791175    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99da311f-1d48-4128-b69a-94b50a66a807-istio-token\") pod \"99da311f-1d48-4128-b69a-94b50a66a807\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") "
Jul 10 04:02:37.791292 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.791280    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99da311f-1d48-4128-b69a-94b50a66a807-istio-podinfo\") pod \"99da311f-1d48-4128-b69a-94b50a66a807\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") "
Jul 10 04:02:37.791400 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.791387    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j879q\" (UniqueName: \"kubernetes.io/projected/99da311f-1d48-4128-b69a-94b50a66a807-kube-api-access-j879q\") pod \"99da311f-1d48-4128-b69a-94b50a66a807\" (UID: \"99da311f-1d48-4128-b69a-94b50a66a807\") "
Jul 10 04:02:37.792044 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.792002    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-credential-socket\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:02:37.797134 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.791217    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-istio-data" (OuterVolumeSpecName: "istio-data") pod "99da311f-1d48-4128-b69a-94b50a66a807" (UID: "99da311f-1d48-4128-b69a-94b50a66a807"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:02:37.797264 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.791590    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99da311f-1d48-4128-b69a-94b50a66a807-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "99da311f-1d48-4128-b69a-94b50a66a807" (UID: "99da311f-1d48-4128-b69a-94b50a66a807"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:02:37.797264 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.791808    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "99da311f-1d48-4128-b69a-94b50a66a807" (UID: "99da311f-1d48-4128-b69a-94b50a66a807"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:02:37.797264 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.794674    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99da311f-1d48-4128-b69a-94b50a66a807-istio-token" (OuterVolumeSpecName: "istio-token") pod "99da311f-1d48-4128-b69a-94b50a66a807" (UID: "99da311f-1d48-4128-b69a-94b50a66a807"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:02:37.797264 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.794863    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "99da311f-1d48-4128-b69a-94b50a66a807" (UID: "99da311f-1d48-4128-b69a-94b50a66a807"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:02:37.797712 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.797681    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99da311f-1d48-4128-b69a-94b50a66a807-kube-api-access-j879q" (OuterVolumeSpecName: "kube-api-access-j879q") pod "99da311f-1d48-4128-b69a-94b50a66a807" (UID: "99da311f-1d48-4128-b69a-94b50a66a807"). InnerVolumeSpecName "kube-api-access-j879q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:02:37.798061 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.798041    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "99da311f-1d48-4128-b69a-94b50a66a807" (UID: "99da311f-1d48-4128-b69a-94b50a66a807"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:02:37.799092 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.799063    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99da311f-1d48-4128-b69a-94b50a66a807-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "99da311f-1d48-4128-b69a-94b50a66a807" (UID: "99da311f-1d48-4128-b69a-94b50a66a807"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 04:02:37.893391 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.893352    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-istio-data\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:02:37.893391 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.893384    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-workload-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:02:37.893391 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.893395    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99da311f-1d48-4128-b69a-94b50a66a807-istiod-ca-cert\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:02:37.893391 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.893403    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-workload-socket\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:02:37.893760 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.893413    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99da311f-1d48-4128-b69a-94b50a66a807-istio-envoy\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:02:37.893760 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.893421    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99da311f-1d48-4128-b69a-94b50a66a807-istio-token\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:02:37.893760 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.893430    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99da311f-1d48-4128-b69a-94b50a66a807-istio-podinfo\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:02:37.893760 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.893439    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j879q\" (UniqueName: \"kubernetes.io/projected/99da311f-1d48-4128-b69a-94b50a66a807-kube-api-access-j879q\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:02:37.964593 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.964509    2571 generic.go:356] "Generic (PLEG): container finished" podID="99da311f-1d48-4128-b69a-94b50a66a807" containerID="af51b045818c6c7a3033e87669c4a854d75c9c6486645b7658d4ffd91cf68884" exitCode=0
Jul 10 04:02:37.964593 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.964575    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"
Jul 10 04:02:37.964767 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.964593    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n" event={"ID":"99da311f-1d48-4128-b69a-94b50a66a807","Type":"ContainerDied","Data":"af51b045818c6c7a3033e87669c4a854d75c9c6486645b7658d4ffd91cf68884"}
Jul 10 04:02:37.964767 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.964674    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n" event={"ID":"99da311f-1d48-4128-b69a-94b50a66a807","Type":"ContainerDied","Data":"901f7ba077517af0542dacc24b74cf053d26de5e8744e92b5cd2b40a43a67417"}
Jul 10 04:02:37.964767 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.964697    2571 scope.go:117] "RemoveContainer" containerID="af51b045818c6c7a3033e87669c4a854d75c9c6486645b7658d4ffd91cf68884"
Jul 10 04:02:37.973783 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.973763    2571 scope.go:117] "RemoveContainer" containerID="6ff5119ed1c31b652665803f241a105eb85fcd984ac1676249f0a078f9a1f0c1"
Jul 10 04:02:37.981556 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.981438    2571 scope.go:117] "RemoveContainer" containerID="af51b045818c6c7a3033e87669c4a854d75c9c6486645b7658d4ffd91cf68884"
Jul 10 04:02:37.981756 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:02:37.981737    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af51b045818c6c7a3033e87669c4a854d75c9c6486645b7658d4ffd91cf68884\": container with ID starting with af51b045818c6c7a3033e87669c4a854d75c9c6486645b7658d4ffd91cf68884 not found: ID does not exist" containerID="af51b045818c6c7a3033e87669c4a854d75c9c6486645b7658d4ffd91cf68884"
Jul 10 04:02:37.981834 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.981769    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af51b045818c6c7a3033e87669c4a854d75c9c6486645b7658d4ffd91cf68884"} err="failed to get container status \"af51b045818c6c7a3033e87669c4a854d75c9c6486645b7658d4ffd91cf68884\": rpc error: code = NotFound desc = could not find container \"af51b045818c6c7a3033e87669c4a854d75c9c6486645b7658d4ffd91cf68884\": container with ID starting with af51b045818c6c7a3033e87669c4a854d75c9c6486645b7658d4ffd91cf68884 not found: ID does not exist"
Jul 10 04:02:37.981834 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.981793    2571 scope.go:117] "RemoveContainer" containerID="6ff5119ed1c31b652665803f241a105eb85fcd984ac1676249f0a078f9a1f0c1"
Jul 10 04:02:37.982081 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:02:37.982064    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ff5119ed1c31b652665803f241a105eb85fcd984ac1676249f0a078f9a1f0c1\": container with ID starting with 6ff5119ed1c31b652665803f241a105eb85fcd984ac1676249f0a078f9a1f0c1 not found: ID does not exist" containerID="6ff5119ed1c31b652665803f241a105eb85fcd984ac1676249f0a078f9a1f0c1"
Jul 10 04:02:37.982124 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.982088    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff5119ed1c31b652665803f241a105eb85fcd984ac1676249f0a078f9a1f0c1"} err="failed to get container status \"6ff5119ed1c31b652665803f241a105eb85fcd984ac1676249f0a078f9a1f0c1\": rpc error: code = NotFound desc = could not find container \"6ff5119ed1c31b652665803f241a105eb85fcd984ac1676249f0a078f9a1f0c1\": container with ID starting with 6ff5119ed1c31b652665803f241a105eb85fcd984ac1676249f0a078f9a1f0c1 not found: ID does not exist"
Jul 10 04:02:37.984880 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.984858    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"]
Jul 10 04:02:37.987952 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:37.987933    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8d8a2105-openshift-default-5894f7746f-9wq7n"]
Jul 10 04:02:39.127061 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:02:39.126988    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99da311f-1d48-4128-b69a-94b50a66a807" path="/var/lib/kubelet/pods/99da311f-1d48-4128-b69a-94b50a66a807/volumes"
Jul 10 04:06:49.973824 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:49.973791    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"]
Jul 10 04:06:49.976301 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:49.974198    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99da311f-1d48-4128-b69a-94b50a66a807" containerName="istio-proxy"
Jul 10 04:06:49.976301 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:49.974213    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="99da311f-1d48-4128-b69a-94b50a66a807" containerName="istio-proxy"
Jul 10 04:06:49.976301 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:49.974228    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99da311f-1d48-4128-b69a-94b50a66a807" containerName="istio-proxy"
Jul 10 04:06:49.976301 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:49.974236    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="99da311f-1d48-4128-b69a-94b50a66a807" containerName="istio-proxy"
Jul 10 04:06:49.976301 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:49.974338    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="99da311f-1d48-4128-b69a-94b50a66a807" containerName="istio-proxy"
Jul 10 04:06:49.976301 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:49.974355    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="99da311f-1d48-4128-b69a-94b50a66a807" containerName="istio-proxy"
Jul 10 04:06:49.977311 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:49.977294    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:49.979747 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:49.979725    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-010d76e9\"/\"openshift-service-ca.crt\""
Jul 10 04:06:49.980536 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:49.980517    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-010d76e9\"/\"default-dockercfg-9mf9p\""
Jul 10 04:06:49.980633 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:49.980517    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-010d76e9\"/\"kube-root-ca.crt\""
Jul 10 04:06:49.980676 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:49.980647    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-010d76e9\"/\"e2e-lifecycle-model-010d76e9-kserve-self-signed-certs\""
Jul 10 04:06:49.987838 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:49.987812    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"]
Jul 10 04:06:50.119126 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.119077    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75j2b\" (UniqueName: \"kubernetes.io/projected/dbcb7db9-08d4-47a1-973b-3653634ed1eb-kube-api-access-75j2b\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.119126 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.119128    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbcb7db9-08d4-47a1-973b-3653634ed1eb-tls-certs\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.119392 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.119267    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-tmp-dir\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.119392 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.119328    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-home\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.119392 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.119382    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-model-cache\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.119599 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.119434    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-dshm\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.220829 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.220794    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-model-cache\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.221037 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.220842    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-dshm\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.221037 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.220888    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75j2b\" (UniqueName: \"kubernetes.io/projected/dbcb7db9-08d4-47a1-973b-3653634ed1eb-kube-api-access-75j2b\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.221037 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.220915    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbcb7db9-08d4-47a1-973b-3653634ed1eb-tls-certs\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.221037 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.220997    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-tmp-dir\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.221303 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.221280    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-model-cache\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.221424 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.221297    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-tmp-dir\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.221516 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.221444    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-home\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.221784 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.221761    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-home\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.223907 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.223846    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-dshm\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.224222 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.224200    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbcb7db9-08d4-47a1-973b-3653634ed1eb-tls-certs\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.228757 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.228733    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75j2b\" (UniqueName: \"kubernetes.io/projected/dbcb7db9-08d4-47a1-973b-3653634ed1eb-kube-api-access-75j2b\") pod \"e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") " pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.287180 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.287135    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:06:50.428140 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:50.428081    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"]
Jul 10 04:06:50.430740 ip-10-0-143-142 kubenswrapper[2571]: W0710 04:06:50.430711    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddbcb7db9_08d4_47a1_973b_3653634ed1eb.slice/crio-f4ce9ea394a67497f396ae3e0119851e0580f5a15bb8ed95187ce1291d6448a6 WatchSource:0}: Error finding container f4ce9ea394a67497f396ae3e0119851e0580f5a15bb8ed95187ce1291d6448a6: Status 404 returned error can't find the container with id f4ce9ea394a67497f396ae3e0119851e0580f5a15bb8ed95187ce1291d6448a6
Jul 10 04:06:51.144477 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:51.144422    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf" event={"ID":"dbcb7db9-08d4-47a1-973b-3653634ed1eb","Type":"ContainerStarted","Data":"ac0fe784a29171de29182fb4320de8bf1a38635808b9858f729ae85ff0922aa2"}
Jul 10 04:06:51.144920 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:51.144490    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf" event={"ID":"dbcb7db9-08d4-47a1-973b-3653634ed1eb","Type":"ContainerStarted","Data":"f4ce9ea394a67497f396ae3e0119851e0580f5a15bb8ed95187ce1291d6448a6"}
Jul 10 04:06:51.159744 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:06:51.159690    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf" podStartSLOduration=2.159674708 podStartE2EDuration="2.159674708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:06:51.159107501 +0000 UTC m=+2046.673842816" watchObservedRunningTime="2026-07-10 04:06:51.159674708 +0000 UTC m=+2046.674410010"
Jul 10 04:07:00.287919 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:00.287867    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:07:00.287919 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:00.287925    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:07:00.300430 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:00.300400    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:07:01.202254 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:01.202220    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:07:30.295394 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:30.295306    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"]
Jul 10 04:07:45.317591 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.317530    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf" podUID="dbcb7db9-08d4-47a1-973b-3653634ed1eb" containerName="main" containerID="cri-o://ac0fe784a29171de29182fb4320de8bf1a38635808b9858f729ae85ff0922aa2" gracePeriod=45
Jul 10 04:07:45.575855 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.575788    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:07:45.738834 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.738793    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-model-cache\") pod \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") "
Jul 10 04:07:45.739032 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.738857    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75j2b\" (UniqueName: \"kubernetes.io/projected/dbcb7db9-08d4-47a1-973b-3653634ed1eb-kube-api-access-75j2b\") pod \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") "
Jul 10 04:07:45.739032 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.738919    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbcb7db9-08d4-47a1-973b-3653634ed1eb-tls-certs\") pod \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") "
Jul 10 04:07:45.739032 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.738939    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-home\") pod \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") "
Jul 10 04:07:45.739032 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.738957    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-dshm\") pod \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") "
Jul 10 04:07:45.739032 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.738986    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-tmp-dir\") pod \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\" (UID: \"dbcb7db9-08d4-47a1-973b-3653634ed1eb\") "
Jul 10 04:07:45.739293 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.739149    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-model-cache" (OuterVolumeSpecName: "model-cache") pod "dbcb7db9-08d4-47a1-973b-3653634ed1eb" (UID: "dbcb7db9-08d4-47a1-973b-3653634ed1eb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:07:45.739293 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.739226    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-home" (OuterVolumeSpecName: "home") pod "dbcb7db9-08d4-47a1-973b-3653634ed1eb" (UID: "dbcb7db9-08d4-47a1-973b-3653634ed1eb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:07:45.739400 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.739298    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dbcb7db9-08d4-47a1-973b-3653634ed1eb" (UID: "dbcb7db9-08d4-47a1-973b-3653634ed1eb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:07:45.739400 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.739310    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-model-cache\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:07:45.741198 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.741164    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-dshm" (OuterVolumeSpecName: "dshm") pod "dbcb7db9-08d4-47a1-973b-3653634ed1eb" (UID: "dbcb7db9-08d4-47a1-973b-3653634ed1eb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:07:45.741198 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.741184    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbcb7db9-08d4-47a1-973b-3653634ed1eb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dbcb7db9-08d4-47a1-973b-3653634ed1eb" (UID: "dbcb7db9-08d4-47a1-973b-3653634ed1eb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:07:45.741348 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.741199    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbcb7db9-08d4-47a1-973b-3653634ed1eb-kube-api-access-75j2b" (OuterVolumeSpecName: "kube-api-access-75j2b") pod "dbcb7db9-08d4-47a1-973b-3653634ed1eb" (UID: "dbcb7db9-08d4-47a1-973b-3653634ed1eb"). InnerVolumeSpecName "kube-api-access-75j2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:07:45.840564 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.840432    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbcb7db9-08d4-47a1-973b-3653634ed1eb-tls-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:07:45.840564 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.840502    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-home\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:07:45.840564 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.840511    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-dshm\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:07:45.840564 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.840536    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbcb7db9-08d4-47a1-973b-3653634ed1eb-tmp-dir\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:07:45.840564 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:45.840546    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75j2b\" (UniqueName: \"kubernetes.io/projected/dbcb7db9-08d4-47a1-973b-3653634ed1eb-kube-api-access-75j2b\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:07:46.402167 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:46.402134    2571 generic.go:356] "Generic (PLEG): container finished" podID="dbcb7db9-08d4-47a1-973b-3653634ed1eb" containerID="ac0fe784a29171de29182fb4320de8bf1a38635808b9858f729ae85ff0922aa2" exitCode=0
Jul 10 04:07:46.402593 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:46.402182    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"
Jul 10 04:07:46.402593 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:46.402219    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf" event={"ID":"dbcb7db9-08d4-47a1-973b-3653634ed1eb","Type":"ContainerDied","Data":"ac0fe784a29171de29182fb4320de8bf1a38635808b9858f729ae85ff0922aa2"}
Jul 10 04:07:46.402593 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:46.402267    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf" event={"ID":"dbcb7db9-08d4-47a1-973b-3653634ed1eb","Type":"ContainerDied","Data":"f4ce9ea394a67497f396ae3e0119851e0580f5a15bb8ed95187ce1291d6448a6"}
Jul 10 04:07:46.402593 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:46.402296    2571 scope.go:117] "RemoveContainer" containerID="ac0fe784a29171de29182fb4320de8bf1a38635808b9858f729ae85ff0922aa2"
Jul 10 04:07:46.412105 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:46.412079    2571 scope.go:117] "RemoveContainer" containerID="ac0fe784a29171de29182fb4320de8bf1a38635808b9858f729ae85ff0922aa2"
Jul 10 04:07:46.412376 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:07:46.412359    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac0fe784a29171de29182fb4320de8bf1a38635808b9858f729ae85ff0922aa2\": container with ID starting with ac0fe784a29171de29182fb4320de8bf1a38635808b9858f729ae85ff0922aa2 not found: ID does not exist" containerID="ac0fe784a29171de29182fb4320de8bf1a38635808b9858f729ae85ff0922aa2"
Jul 10 04:07:46.412421 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:46.412386    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac0fe784a29171de29182fb4320de8bf1a38635808b9858f729ae85ff0922aa2"} err="failed to get container status \"ac0fe784a29171de29182fb4320de8bf1a38635808b9858f729ae85ff0922aa2\": rpc error: code = NotFound desc = could not find container \"ac0fe784a29171de29182fb4320de8bf1a38635808b9858f729ae85ff0922aa2\": container with ID starting with ac0fe784a29171de29182fb4320de8bf1a38635808b9858f729ae85ff0922aa2 not found: ID does not exist"
Jul 10 04:07:46.425486 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:46.425442    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"]
Jul 10 04:07:46.429007 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:46.428975    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-010d76e9/e2e-lifecycle-model-010d76e9-kserve-ff6fc6cf5-xmpvf"]
Jul 10 04:07:47.126485 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:07:47.126426    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbcb7db9-08d4-47a1-973b-3653634ed1eb" path="/var/lib/kubelet/pods/dbcb7db9-08d4-47a1-973b-3653634ed1eb/volumes"
Jul 10 04:09:19.797335 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.797295    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"]
Jul 10 04:09:19.797842 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.797702    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbcb7db9-08d4-47a1-973b-3653634ed1eb" containerName="main"
Jul 10 04:09:19.797842 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.797714    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbcb7db9-08d4-47a1-973b-3653634ed1eb" containerName="main"
Jul 10 04:09:19.797842 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.797775    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbcb7db9-08d4-47a1-973b-3653634ed1eb" containerName="main"
Jul 10 04:09:19.800656 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.800632    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:19.804955 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.804928    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-7fe142-openshift-default-dockercfg-8mwjz\""
Jul 10 04:09:19.821098 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.821064    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"]
Jul 10 04:09:19.976181 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.976141    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-token\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:19.976181 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.976189    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istiod-ca-cert\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:19.976501 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.976264    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25qh8\" (UniqueName: \"kubernetes.io/projected/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-kube-api-access-25qh8\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:19.976501 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.976313    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-workload-certs\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:19.976501 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.976342    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-podinfo\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:19.976629 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.976510    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-envoy\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:19.976629 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.976557    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-data\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:19.976698 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.976645    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-workload-socket\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:19.976698 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:19.976674    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-credential-socket\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.077973 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.077870    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-envoy\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.077973 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.077925    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-data\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.077973 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.077962    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-workload-socket\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.078262 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.077994    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-credential-socket\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.078262 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.078021    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-token\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.078262 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.078044    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istiod-ca-cert\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.078262 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.078076    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25qh8\" (UniqueName: \"kubernetes.io/projected/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-kube-api-access-25qh8\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.078262 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.078103    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-workload-certs\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.078262 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.078125    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-podinfo\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.078623 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.078441    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-workload-socket\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.078623 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.078524    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-credential-socket\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.078778 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.078749    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-workload-certs\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.078913 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.078790    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-data\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.078981 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.078911    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istiod-ca-cert\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.080318 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.080297    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-envoy\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.080615 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.080596    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-podinfo\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.085865 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.085839    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-token\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.085998 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.085978    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25qh8\" (UniqueName: \"kubernetes.io/projected/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-kube-api-access-25qh8\") pod \"e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") " pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.111529 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.111497    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:20.276618 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.276590    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"]
Jul 10 04:09:20.278707 ip-10-0-143-142 kubenswrapper[2571]: W0710 04:09:20.278677    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d3c8c1f_274a_4b06_b591_a80bd361d0c0.slice/crio-02a28c0319a8165f5933ca716778e7f871d7bef2c7af9b4fb93ac4138fe6db04 WatchSource:0}: Error finding container 02a28c0319a8165f5933ca716778e7f871d7bef2c7af9b4fb93ac4138fe6db04: Status 404 returned error can't find the container with id 02a28c0319a8165f5933ca716778e7f871d7bef2c7af9b4fb93ac4138fe6db04
Jul 10 04:09:20.280810 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.280775    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:09:20.280892 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.280847    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:09:20.280892 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.280876    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:09:20.857190 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.857152    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp" event={"ID":"2d3c8c1f-274a-4b06-b591-a80bd361d0c0","Type":"ContainerStarted","Data":"4de0d8a94bf8d75ececb2060f2f867b1172fae389a8f601cb98a6e00e6137583"}
Jul 10 04:09:20.857702 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.857254    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp" event={"ID":"2d3c8c1f-274a-4b06-b591-a80bd361d0c0","Type":"ContainerStarted","Data":"02a28c0319a8165f5933ca716778e7f871d7bef2c7af9b4fb93ac4138fe6db04"}
Jul 10 04:09:20.877765 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:20.877692    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp" podStartSLOduration=1.8776696510000002 podStartE2EDuration="1.877669651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:09:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:09:20.876881419 +0000 UTC m=+2196.391616723" watchObservedRunningTime="2026-07-10 04:09:20.877669651 +0000 UTC m=+2196.392404955"
Jul 10 04:09:21.112064 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:21.111970    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:21.117239 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:21.117211    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:21.862362 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:21.862330    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:21.863550 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:21.863528    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:37.903689 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:37.903647    2571 patch_prober.go:28] interesting pod/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 04:09:37.904135 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:37.903715    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp" podUID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 04:09:38.671146 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.671030    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"]
Jul 10 04:09:38.674541 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.674517    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.677169 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.676959    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-72703e\"/\"openshift-service-ca.crt\""
Jul 10 04:09:38.677169 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.677048    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-72703e\"/\"kube-root-ca.crt\""
Jul 10 04:09:38.678405 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.678012    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-72703e\"/\"test-model-72703e-kserve-self-signed-certs\""
Jul 10 04:09:38.678405 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.678106    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-72703e\"/\"default-dockercfg-vmjv7\""
Jul 10 04:09:38.695776 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.695739    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"]
Jul 10 04:09:38.766992 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.766956    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-home\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.767157 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.767018    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-model-cache\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.767157 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.767043    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-dshm\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.767157 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.767058    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-tmp-dir\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.767157 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.767077    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rztzk\" (UniqueName: \"kubernetes.io/projected/73e10af0-7cc0-4038-9d78-45a2615320e2-kube-api-access-rztzk\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.767291 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.767207    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73e10af0-7cc0-4038-9d78-45a2615320e2-tls-certs\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.868029 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.867988    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-model-cache\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.868029 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.868032    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-dshm\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.868288 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.868054    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-tmp-dir\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.868288 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.868080    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rztzk\" (UniqueName: \"kubernetes.io/projected/73e10af0-7cc0-4038-9d78-45a2615320e2-kube-api-access-rztzk\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.868288 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.868120    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73e10af0-7cc0-4038-9d78-45a2615320e2-tls-certs\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.868288 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.868233    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-home\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.868527 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.868497    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-tmp-dir\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.868610 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.868590    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-model-cache\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.868663 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.868607    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-home\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.870564 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.870537    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-dshm\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.870865 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.870839    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73e10af0-7cc0-4038-9d78-45a2615320e2-tls-certs\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.876646 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.876580    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rztzk\" (UniqueName: \"kubernetes.io/projected/73e10af0-7cc0-4038-9d78-45a2615320e2-kube-api-access-rztzk\") pod \"test-model-72703e-kserve-59696cbf9c-6tc7w\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") " pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:38.946353 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.946255    2571 generic.go:356] "Generic (PLEG): container finished" podID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerID="4de0d8a94bf8d75ececb2060f2f867b1172fae389a8f601cb98a6e00e6137583" exitCode=0
Jul 10 04:09:38.946353 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.946314    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp" event={"ID":"2d3c8c1f-274a-4b06-b591-a80bd361d0c0","Type":"ContainerDied","Data":"4de0d8a94bf8d75ececb2060f2f867b1172fae389a8f601cb98a6e00e6137583"}
Jul 10 04:09:38.946920 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.946901    2571 scope.go:117] "RemoveContainer" containerID="4de0d8a94bf8d75ececb2060f2f867b1172fae389a8f601cb98a6e00e6137583"
Jul 10 04:09:38.948180 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.948146    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:09:38.948291 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.948233    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:09:38.948291 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.948264    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:09:38.994663 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:38.994638    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:39.153907 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:39.153872    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"]
Jul 10 04:09:39.157153 ip-10-0-143-142 kubenswrapper[2571]: W0710 04:09:39.157112    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73e10af0_7cc0_4038_9d78_45a2615320e2.slice/crio-395a67d1b1d734f2f1533c20af2ffc0f91e314c0ba397a284fcf5195a2cb058f WatchSource:0}: Error finding container 395a67d1b1d734f2f1533c20af2ffc0f91e314c0ba397a284fcf5195a2cb058f: Status 404 returned error can't find the container with id 395a67d1b1d734f2f1533c20af2ffc0f91e314c0ba397a284fcf5195a2cb058f
Jul 10 04:09:39.954302 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:39.954256    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w" event={"ID":"73e10af0-7cc0-4038-9d78-45a2615320e2","Type":"ContainerStarted","Data":"4429bc1d994f879187cd3e6b6a81aca61b3beecbceb6731311ef68f81e2ce160"}
Jul 10 04:09:39.954822 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:39.954312    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w" event={"ID":"73e10af0-7cc0-4038-9d78-45a2615320e2","Type":"ContainerStarted","Data":"395a67d1b1d734f2f1533c20af2ffc0f91e314c0ba397a284fcf5195a2cb058f"}
Jul 10 04:09:39.956445 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:39.956414    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp" event={"ID":"2d3c8c1f-274a-4b06-b591-a80bd361d0c0","Type":"ContainerStarted","Data":"ff9ed6d83e69e47ace59081b4d54e0d3d1c1610a19e80ced6b4be9a0fc9a821b"}
Jul 10 04:09:39.972633 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:39.972584    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w" podStartSLOduration=1.972566927 podStartE2EDuration="1.972566927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:09:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:09:39.970582705 +0000 UTC m=+2215.485318003" watchObservedRunningTime="2026-07-10 04:09:39.972566927 +0000 UTC m=+2215.487302230"
Jul 10 04:09:40.112175 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:40.112139    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:40.112370 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:40.112344    2571 patch_prober.go:28] interesting pod/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 10 04:09:40.112451 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:40.112409    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp" podUID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 10 04:09:41.112034 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:41.111992    2571 patch_prober.go:28] interesting pod/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 10 04:09:41.112430 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:41.112052    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp" podUID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 10 04:09:42.112391 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:42.112353    2571 patch_prober.go:28] interesting pod/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 10 04:09:42.112794 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:42.112424    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp" podUID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 10 04:09:43.112228 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:43.112187    2571 patch_prober.go:28] interesting pod/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 10 04:09:43.112421 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:43.112244    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp" podUID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 10 04:09:44.115887 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:44.115853    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:44.116369 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:44.116344    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:44.117218 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:44.117202    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:09:48.995540 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:48.995495    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:48.995980 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:48.995555    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:49.008633 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:49.008604    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:09:49.020935 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:09:49.020908    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:10:09.273044 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.272953    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"]
Jul 10 04:10:09.275552 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.275533    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.277789 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.277765    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7fe142\"/\"openshift-service-ca.crt\""
Jul 10 04:10:09.277912 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.277826    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7fe142\"/\"default-dockercfg-84zv8\""
Jul 10 04:10:09.278629 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.278611    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7fe142\"/\"kube-root-ca.crt\""
Jul 10 04:10:09.278741 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.278675    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7fe142\"/\"test-model-7fe142-kserve-self-signed-certs\""
Jul 10 04:10:09.282904 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.282884    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"]
Jul 10 04:10:09.355633 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.355598    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a681ec-5893-42cd-a1cc-aed597ecd60b-tls-certs\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.355844 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.355648    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-home\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.355844 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.355729    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-model-cache\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.355844 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.355772    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnblq\" (UniqueName: \"kubernetes.io/projected/f8a681ec-5893-42cd-a1cc-aed597ecd60b-kube-api-access-dnblq\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.355844 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.355803    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-tmp-dir\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.355844 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.355832    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-dshm\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.456900 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.456860    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a681ec-5893-42cd-a1cc-aed597ecd60b-tls-certs\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.457079 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.456911    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-home\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.457079 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.456940    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-model-cache\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.457079 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.456967    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnblq\" (UniqueName: \"kubernetes.io/projected/f8a681ec-5893-42cd-a1cc-aed597ecd60b-kube-api-access-dnblq\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.457079 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.457003    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-tmp-dir\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.457079 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.457039    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-dshm\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.457370 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.457347    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-home\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.457434 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.457394    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-model-cache\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.457556 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.457446    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-tmp-dir\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.459449 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.459427    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-dshm\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.459681 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.459664    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a681ec-5893-42cd-a1cc-aed597ecd60b-tls-certs\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.464542 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.464524    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnblq\" (UniqueName: \"kubernetes.io/projected/f8a681ec-5893-42cd-a1cc-aed597ecd60b-kube-api-access-dnblq\") pod \"test-model-7fe142-kserve-65c84b87ff-ls4zz\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") " pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.586177 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.586083    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:09.729233 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:09.729211    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"]
Jul 10 04:10:09.731589 ip-10-0-143-142 kubenswrapper[2571]: W0710 04:10:09.731561    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8a681ec_5893_42cd_a1cc_aed597ecd60b.slice/crio-e2bd97aac3ddf9c1326ff79f07fb9c65fbf4956972a09c3a52c1ee2fe83cd9db WatchSource:0}: Error finding container e2bd97aac3ddf9c1326ff79f07fb9c65fbf4956972a09c3a52c1ee2fe83cd9db: Status 404 returned error can't find the container with id e2bd97aac3ddf9c1326ff79f07fb9c65fbf4956972a09c3a52c1ee2fe83cd9db
Jul 10 04:10:10.117204 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:10.117160    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz" event={"ID":"f8a681ec-5893-42cd-a1cc-aed597ecd60b","Type":"ContainerStarted","Data":"6507e8ad8643d3d922907a4486ba8f9f3fd221f6494c5754a1cd5c77243f8fac"}
Jul 10 04:10:10.117388 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:10.117214    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz" event={"ID":"f8a681ec-5893-42cd-a1cc-aed597ecd60b","Type":"ContainerStarted","Data":"e2bd97aac3ddf9c1326ff79f07fb9c65fbf4956972a09c3a52c1ee2fe83cd9db"}
Jul 10 04:10:10.133434 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:10.133379    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz" podStartSLOduration=1.133362999 podStartE2EDuration="1.133362999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:10:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:10:10.131760428 +0000 UTC m=+2245.646495722" watchObservedRunningTime="2026-07-10 04:10:10.133362999 +0000 UTC m=+2245.648098302"
Jul 10 04:10:19.587094 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:19.587049    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:19.587520 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:19.587110    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:19.599910 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:19.599877    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:20.175653 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:20.175625    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:36.575269 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:36.575235    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"]
Jul 10 04:10:37.516569 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:37.516536    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"]
Jul 10 04:10:51.596230 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.596182    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w" podUID="73e10af0-7cc0-4038-9d78-45a2615320e2" containerName="main" containerID="cri-o://4429bc1d994f879187cd3e6b6a81aca61b3beecbceb6731311ef68f81e2ce160" gracePeriod=45
Jul 10 04:10:51.851519 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.851435    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:10:51.861284 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.861253    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73e10af0-7cc0-4038-9d78-45a2615320e2-tls-certs\") pod \"73e10af0-7cc0-4038-9d78-45a2615320e2\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") "
Jul 10 04:10:51.861426 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.861338    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-model-cache\") pod \"73e10af0-7cc0-4038-9d78-45a2615320e2\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") "
Jul 10 04:10:51.861426 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.861390    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-home\") pod \"73e10af0-7cc0-4038-9d78-45a2615320e2\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") "
Jul 10 04:10:51.861515 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.861431    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rztzk\" (UniqueName: \"kubernetes.io/projected/73e10af0-7cc0-4038-9d78-45a2615320e2-kube-api-access-rztzk\") pod \"73e10af0-7cc0-4038-9d78-45a2615320e2\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") "
Jul 10 04:10:51.861515 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.861509    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-tmp-dir\") pod \"73e10af0-7cc0-4038-9d78-45a2615320e2\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") "
Jul 10 04:10:51.861619 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.861542    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-dshm\") pod \"73e10af0-7cc0-4038-9d78-45a2615320e2\" (UID: \"73e10af0-7cc0-4038-9d78-45a2615320e2\") "
Jul 10 04:10:51.861619 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.861573    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "73e10af0-7cc0-4038-9d78-45a2615320e2" (UID: "73e10af0-7cc0-4038-9d78-45a2615320e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:10:51.861722 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.861661    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-home" (OuterVolumeSpecName: "home") pod "73e10af0-7cc0-4038-9d78-45a2615320e2" (UID: "73e10af0-7cc0-4038-9d78-45a2615320e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:10:51.861802 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.861781    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "73e10af0-7cc0-4038-9d78-45a2615320e2" (UID: "73e10af0-7cc0-4038-9d78-45a2615320e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:10:51.861914 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.861889    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-tmp-dir\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:10:51.861914 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.861915    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-model-cache\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:10:51.862108 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.861926    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-home\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:10:51.863621 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.863597    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-dshm" (OuterVolumeSpecName: "dshm") pod "73e10af0-7cc0-4038-9d78-45a2615320e2" (UID: "73e10af0-7cc0-4038-9d78-45a2615320e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:10:51.863723 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.863684    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73e10af0-7cc0-4038-9d78-45a2615320e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "73e10af0-7cc0-4038-9d78-45a2615320e2" (UID: "73e10af0-7cc0-4038-9d78-45a2615320e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:10:51.863796 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.863771    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73e10af0-7cc0-4038-9d78-45a2615320e2-kube-api-access-rztzk" (OuterVolumeSpecName: "kube-api-access-rztzk") pod "73e10af0-7cc0-4038-9d78-45a2615320e2" (UID: "73e10af0-7cc0-4038-9d78-45a2615320e2"). InnerVolumeSpecName "kube-api-access-rztzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:10:51.963507 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.963439    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rztzk\" (UniqueName: \"kubernetes.io/projected/73e10af0-7cc0-4038-9d78-45a2615320e2-kube-api-access-rztzk\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:10:51.963507 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.963510    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73e10af0-7cc0-4038-9d78-45a2615320e2-dshm\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:10:51.963705 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:51.963527    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73e10af0-7cc0-4038-9d78-45a2615320e2-tls-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:10:52.324261 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.324224    2571 generic.go:356] "Generic (PLEG): container finished" podID="73e10af0-7cc0-4038-9d78-45a2615320e2" containerID="4429bc1d994f879187cd3e6b6a81aca61b3beecbceb6731311ef68f81e2ce160" exitCode=0
Jul 10 04:10:52.324261 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.324255    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w" event={"ID":"73e10af0-7cc0-4038-9d78-45a2615320e2","Type":"ContainerDied","Data":"4429bc1d994f879187cd3e6b6a81aca61b3beecbceb6731311ef68f81e2ce160"}
Jul 10 04:10:52.324537 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.324273    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"
Jul 10 04:10:52.324537 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.324304    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w" event={"ID":"73e10af0-7cc0-4038-9d78-45a2615320e2","Type":"ContainerDied","Data":"395a67d1b1d734f2f1533c20af2ffc0f91e314c0ba397a284fcf5195a2cb058f"}
Jul 10 04:10:52.324537 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.324324    2571 scope.go:117] "RemoveContainer" containerID="4429bc1d994f879187cd3e6b6a81aca61b3beecbceb6731311ef68f81e2ce160"
Jul 10 04:10:52.334100 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.334076    2571 scope.go:117] "RemoveContainer" containerID="4429bc1d994f879187cd3e6b6a81aca61b3beecbceb6731311ef68f81e2ce160"
Jul 10 04:10:52.334420 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:10:52.334400    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4429bc1d994f879187cd3e6b6a81aca61b3beecbceb6731311ef68f81e2ce160\": container with ID starting with 4429bc1d994f879187cd3e6b6a81aca61b3beecbceb6731311ef68f81e2ce160 not found: ID does not exist" containerID="4429bc1d994f879187cd3e6b6a81aca61b3beecbceb6731311ef68f81e2ce160"
Jul 10 04:10:52.334493 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.334430    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4429bc1d994f879187cd3e6b6a81aca61b3beecbceb6731311ef68f81e2ce160"} err="failed to get container status \"4429bc1d994f879187cd3e6b6a81aca61b3beecbceb6731311ef68f81e2ce160\": rpc error: code = NotFound desc = could not find container \"4429bc1d994f879187cd3e6b6a81aca61b3beecbceb6731311ef68f81e2ce160\": container with ID starting with 4429bc1d994f879187cd3e6b6a81aca61b3beecbceb6731311ef68f81e2ce160 not found: ID does not exist"
Jul 10 04:10:52.345600 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.345567    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"]
Jul 10 04:10:52.349009 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.348982    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-72703e/test-model-72703e-kserve-59696cbf9c-6tc7w"]
Jul 10 04:10:52.541390 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.541340    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz" podUID="f8a681ec-5893-42cd-a1cc-aed597ecd60b" containerName="main" containerID="cri-o://6507e8ad8643d3d922907a4486ba8f9f3fd221f6494c5754a1cd5c77243f8fac" gracePeriod=45
Jul 10 04:10:52.796838 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.796809    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:52.872067 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.871986    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-model-cache\") pod \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") "
Jul 10 04:10:52.872067 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.872020    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-dshm\") pod \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") "
Jul 10 04:10:52.872067 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.872057    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-tmp-dir\") pod \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") "
Jul 10 04:10:52.872722 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.872106    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-home\") pod \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") "
Jul 10 04:10:52.872722 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.872140    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnblq\" (UniqueName: \"kubernetes.io/projected/f8a681ec-5893-42cd-a1cc-aed597ecd60b-kube-api-access-dnblq\") pod \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") "
Jul 10 04:10:52.872722 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.872164    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a681ec-5893-42cd-a1cc-aed597ecd60b-tls-certs\") pod \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\" (UID: \"f8a681ec-5893-42cd-a1cc-aed597ecd60b\") "
Jul 10 04:10:52.872722 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.872270    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-model-cache" (OuterVolumeSpecName: "model-cache") pod "f8a681ec-5893-42cd-a1cc-aed597ecd60b" (UID: "f8a681ec-5893-42cd-a1cc-aed597ecd60b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:10:52.872722 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.872355    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f8a681ec-5893-42cd-a1cc-aed597ecd60b" (UID: "f8a681ec-5893-42cd-a1cc-aed597ecd60b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:10:52.872722 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.872376    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-home" (OuterVolumeSpecName: "home") pod "f8a681ec-5893-42cd-a1cc-aed597ecd60b" (UID: "f8a681ec-5893-42cd-a1cc-aed597ecd60b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:10:52.872722 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.872406    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-model-cache\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:10:52.874339 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.874314    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8a681ec-5893-42cd-a1cc-aed597ecd60b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f8a681ec-5893-42cd-a1cc-aed597ecd60b" (UID: "f8a681ec-5893-42cd-a1cc-aed597ecd60b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:10:52.874415 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.874319    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a681ec-5893-42cd-a1cc-aed597ecd60b-kube-api-access-dnblq" (OuterVolumeSpecName: "kube-api-access-dnblq") pod "f8a681ec-5893-42cd-a1cc-aed597ecd60b" (UID: "f8a681ec-5893-42cd-a1cc-aed597ecd60b"). InnerVolumeSpecName "kube-api-access-dnblq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:10:52.874415 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.874386    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-dshm" (OuterVolumeSpecName: "dshm") pod "f8a681ec-5893-42cd-a1cc-aed597ecd60b" (UID: "f8a681ec-5893-42cd-a1cc-aed597ecd60b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:10:52.973564 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.973530    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-home\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:10:52.973564 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.973559    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnblq\" (UniqueName: \"kubernetes.io/projected/f8a681ec-5893-42cd-a1cc-aed597ecd60b-kube-api-access-dnblq\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:10:52.973564 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.973572    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a681ec-5893-42cd-a1cc-aed597ecd60b-tls-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:10:52.973864 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.973581    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-dshm\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:10:52.973864 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:52.973589    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a681ec-5893-42cd-a1cc-aed597ecd60b-tmp-dir\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:10:53.128669 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:53.128589    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73e10af0-7cc0-4038-9d78-45a2615320e2" path="/var/lib/kubelet/pods/73e10af0-7cc0-4038-9d78-45a2615320e2/volumes"
Jul 10 04:10:53.332406 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:53.332369    2571 generic.go:356] "Generic (PLEG): container finished" podID="f8a681ec-5893-42cd-a1cc-aed597ecd60b" containerID="6507e8ad8643d3d922907a4486ba8f9f3fd221f6494c5754a1cd5c77243f8fac" exitCode=0
Jul 10 04:10:53.332614 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:53.332412    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"
Jul 10 04:10:53.332614 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:53.332449    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz" event={"ID":"f8a681ec-5893-42cd-a1cc-aed597ecd60b","Type":"ContainerDied","Data":"6507e8ad8643d3d922907a4486ba8f9f3fd221f6494c5754a1cd5c77243f8fac"}
Jul 10 04:10:53.332614 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:53.332524    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz" event={"ID":"f8a681ec-5893-42cd-a1cc-aed597ecd60b","Type":"ContainerDied","Data":"e2bd97aac3ddf9c1326ff79f07fb9c65fbf4956972a09c3a52c1ee2fe83cd9db"}
Jul 10 04:10:53.332614 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:53.332551    2571 scope.go:117] "RemoveContainer" containerID="6507e8ad8643d3d922907a4486ba8f9f3fd221f6494c5754a1cd5c77243f8fac"
Jul 10 04:10:53.341289 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:53.341269    2571 scope.go:117] "RemoveContainer" containerID="6507e8ad8643d3d922907a4486ba8f9f3fd221f6494c5754a1cd5c77243f8fac"
Jul 10 04:10:53.341644 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:10:53.341624    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6507e8ad8643d3d922907a4486ba8f9f3fd221f6494c5754a1cd5c77243f8fac\": container with ID starting with 6507e8ad8643d3d922907a4486ba8f9f3fd221f6494c5754a1cd5c77243f8fac not found: ID does not exist" containerID="6507e8ad8643d3d922907a4486ba8f9f3fd221f6494c5754a1cd5c77243f8fac"
Jul 10 04:10:53.341726 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:53.341658    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6507e8ad8643d3d922907a4486ba8f9f3fd221f6494c5754a1cd5c77243f8fac"} err="failed to get container status \"6507e8ad8643d3d922907a4486ba8f9f3fd221f6494c5754a1cd5c77243f8fac\": rpc error: code = NotFound desc = could not find container \"6507e8ad8643d3d922907a4486ba8f9f3fd221f6494c5754a1cd5c77243f8fac\": container with ID starting with 6507e8ad8643d3d922907a4486ba8f9f3fd221f6494c5754a1cd5c77243f8fac not found: ID does not exist"
Jul 10 04:10:53.348077 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:53.348051    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"]
Jul 10 04:10:53.351517 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:53.351490    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-7fe142/test-model-7fe142-kserve-65c84b87ff-ls4zz"]
Jul 10 04:10:55.130711 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:10:55.130678    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8a681ec-5893-42cd-a1cc-aed597ecd60b" path="/var/lib/kubelet/pods/f8a681ec-5893-42cd-a1cc-aed597ecd60b/volumes"
Jul 10 04:11:29.600152 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:29.600114    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"]
Jul 10 04:11:29.600773 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:29.600363    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp" podUID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerName="istio-proxy" containerID="cri-o://ff9ed6d83e69e47ace59081b4d54e0d3d1c1610a19e80ced6b4be9a0fc9a821b" gracePeriod=30
Jul 10 04:11:34.862633 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.862566    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:11:34.959041 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959000    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25qh8\" (UniqueName: \"kubernetes.io/projected/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-kube-api-access-25qh8\") pod \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") "
Jul 10 04:11:34.959297 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959054    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-token\") pod \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") "
Jul 10 04:11:34.959297 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959097    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istiod-ca-cert\") pod \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") "
Jul 10 04:11:34.959297 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959213    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-workload-socket\") pod \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") "
Jul 10 04:11:34.959297 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959266    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-envoy\") pod \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") "
Jul 10 04:11:34.959297 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959292    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-credential-socket\") pod \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") "
Jul 10 04:11:34.959618 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959355    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-data\") pod \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") "
Jul 10 04:11:34.959618 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959388    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-workload-certs\") pod \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") "
Jul 10 04:11:34.959618 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959412    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-podinfo\") pod \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\" (UID: \"2d3c8c1f-274a-4b06-b591-a80bd361d0c0\") "
Jul 10 04:11:34.959618 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959434    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2d3c8c1f-274a-4b06-b591-a80bd361d0c0" (UID: "2d3c8c1f-274a-4b06-b591-a80bd361d0c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:11:34.959618 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959500    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2d3c8c1f-274a-4b06-b591-a80bd361d0c0" (UID: "2d3c8c1f-274a-4b06-b591-a80bd361d0c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:11:34.959880 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959653    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2d3c8c1f-274a-4b06-b591-a80bd361d0c0" (UID: "2d3c8c1f-274a-4b06-b591-a80bd361d0c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:11:34.959880 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959750    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istiod-ca-cert\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:11:34.959880 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959767    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-workload-socket\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:11:34.959880 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.959780    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-workload-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:11:34.960265 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.960238    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "2d3c8c1f-274a-4b06-b591-a80bd361d0c0" (UID: "2d3c8c1f-274a-4b06-b591-a80bd361d0c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:11:34.960380 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.960354    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2d3c8c1f-274a-4b06-b591-a80bd361d0c0" (UID: "2d3c8c1f-274a-4b06-b591-a80bd361d0c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:11:34.961318 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.961293    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-kube-api-access-25qh8" (OuterVolumeSpecName: "kube-api-access-25qh8") pod "2d3c8c1f-274a-4b06-b591-a80bd361d0c0" (UID: "2d3c8c1f-274a-4b06-b591-a80bd361d0c0"). InnerVolumeSpecName "kube-api-access-25qh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:11:34.961415 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.961394    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2d3c8c1f-274a-4b06-b591-a80bd361d0c0" (UID: "2d3c8c1f-274a-4b06-b591-a80bd361d0c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:11:34.961632 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.961614    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "2d3c8c1f-274a-4b06-b591-a80bd361d0c0" (UID: "2d3c8c1f-274a-4b06-b591-a80bd361d0c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:11:34.961740 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:34.961725    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2d3c8c1f-274a-4b06-b591-a80bd361d0c0" (UID: "2d3c8c1f-274a-4b06-b591-a80bd361d0c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 04:11:35.060733 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.060694    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25qh8\" (UniqueName: \"kubernetes.io/projected/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-kube-api-access-25qh8\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:11:35.060733 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.060728    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-token\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:11:35.060733 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.060737    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-envoy\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:11:35.060985 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.060745    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-credential-socket\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:11:35.060985 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.060754    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-data\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:11:35.060985 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.060762    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d3c8c1f-274a-4b06-b591-a80bd361d0c0-istio-podinfo\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:11:35.545647 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.545612    2571 generic.go:356] "Generic (PLEG): container finished" podID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerID="ff9ed6d83e69e47ace59081b4d54e0d3d1c1610a19e80ced6b4be9a0fc9a821b" exitCode=0
Jul 10 04:11:35.545830 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.545660    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"
Jul 10 04:11:35.545830 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.545660    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp" event={"ID":"2d3c8c1f-274a-4b06-b591-a80bd361d0c0","Type":"ContainerDied","Data":"ff9ed6d83e69e47ace59081b4d54e0d3d1c1610a19e80ced6b4be9a0fc9a821b"}
Jul 10 04:11:35.545830 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.545720    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp" event={"ID":"2d3c8c1f-274a-4b06-b591-a80bd361d0c0","Type":"ContainerDied","Data":"02a28c0319a8165f5933ca716778e7f871d7bef2c7af9b4fb93ac4138fe6db04"}
Jul 10 04:11:35.545830 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.545742    2571 scope.go:117] "RemoveContainer" containerID="ff9ed6d83e69e47ace59081b4d54e0d3d1c1610a19e80ced6b4be9a0fc9a821b"
Jul 10 04:11:35.555407 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.555381    2571 scope.go:117] "RemoveContainer" containerID="4de0d8a94bf8d75ececb2060f2f867b1172fae389a8f601cb98a6e00e6137583"
Jul 10 04:11:35.562852 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.562827    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"]
Jul 10 04:11:35.566808 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.566784    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-7fe142-openshift-default-8b65c4b97-dwfdp"]
Jul 10 04:11:35.567154 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.567142    2571 scope.go:117] "RemoveContainer" containerID="ff9ed6d83e69e47ace59081b4d54e0d3d1c1610a19e80ced6b4be9a0fc9a821b"
Jul 10 04:11:35.567353 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:11:35.567327    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff9ed6d83e69e47ace59081b4d54e0d3d1c1610a19e80ced6b4be9a0fc9a821b\": container with ID starting with ff9ed6d83e69e47ace59081b4d54e0d3d1c1610a19e80ced6b4be9a0fc9a821b not found: ID does not exist" containerID="ff9ed6d83e69e47ace59081b4d54e0d3d1c1610a19e80ced6b4be9a0fc9a821b"
Jul 10 04:11:35.567486 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.567362    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff9ed6d83e69e47ace59081b4d54e0d3d1c1610a19e80ced6b4be9a0fc9a821b"} err="failed to get container status \"ff9ed6d83e69e47ace59081b4d54e0d3d1c1610a19e80ced6b4be9a0fc9a821b\": rpc error: code = NotFound desc = could not find container \"ff9ed6d83e69e47ace59081b4d54e0d3d1c1610a19e80ced6b4be9a0fc9a821b\": container with ID starting with ff9ed6d83e69e47ace59081b4d54e0d3d1c1610a19e80ced6b4be9a0fc9a821b not found: ID does not exist"
Jul 10 04:11:35.567486 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.567381    2571 scope.go:117] "RemoveContainer" containerID="4de0d8a94bf8d75ececb2060f2f867b1172fae389a8f601cb98a6e00e6137583"
Jul 10 04:11:35.567759 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:11:35.567734    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4de0d8a94bf8d75ececb2060f2f867b1172fae389a8f601cb98a6e00e6137583\": container with ID starting with 4de0d8a94bf8d75ececb2060f2f867b1172fae389a8f601cb98a6e00e6137583 not found: ID does not exist" containerID="4de0d8a94bf8d75ececb2060f2f867b1172fae389a8f601cb98a6e00e6137583"
Jul 10 04:11:35.567853 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:35.567768    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4de0d8a94bf8d75ececb2060f2f867b1172fae389a8f601cb98a6e00e6137583"} err="failed to get container status \"4de0d8a94bf8d75ececb2060f2f867b1172fae389a8f601cb98a6e00e6137583\": rpc error: code = NotFound desc = could not find container \"4de0d8a94bf8d75ececb2060f2f867b1172fae389a8f601cb98a6e00e6137583\": container with ID starting with 4de0d8a94bf8d75ececb2060f2f867b1172fae389a8f601cb98a6e00e6137583 not found: ID does not exist"
Jul 10 04:11:37.128524 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:37.128407    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" path="/var/lib/kubelet/pods/2d3c8c1f-274a-4b06-b591-a80bd361d0c0/volumes"
Jul 10 04:11:38.731655 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.731617    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"]
Jul 10 04:11:38.732809 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.732778    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73e10af0-7cc0-4038-9d78-45a2615320e2" containerName="main"
Jul 10 04:11:38.732809 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.732806    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="73e10af0-7cc0-4038-9d78-45a2615320e2" containerName="main"
Jul 10 04:11:38.732957 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.732819    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8a681ec-5893-42cd-a1cc-aed597ecd60b" containerName="main"
Jul 10 04:11:38.732957 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.732829    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8a681ec-5893-42cd-a1cc-aed597ecd60b" containerName="main"
Jul 10 04:11:38.732957 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.732855    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerName="istio-proxy"
Jul 10 04:11:38.732957 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.732864    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerName="istio-proxy"
Jul 10 04:11:38.732957 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.732875    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerName="istio-proxy"
Jul 10 04:11:38.732957 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.732883    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerName="istio-proxy"
Jul 10 04:11:38.733146 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.732961    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="73e10af0-7cc0-4038-9d78-45a2615320e2" containerName="main"
Jul 10 04:11:38.733146 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.732973    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8a681ec-5893-42cd-a1cc-aed597ecd60b" containerName="main"
Jul 10 04:11:38.733146 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.732983    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerName="istio-proxy"
Jul 10 04:11:38.733146 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.732994    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d3c8c1f-274a-4b06-b591-a80bd361d0c0" containerName="istio-proxy"
Jul 10 04:11:38.736918 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.736897    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.739040 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.739018    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a335e2-openshift-default-dockercfg-kldb9\""
Jul 10 04:11:38.745640 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.745614    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"]
Jul 10 04:11:38.790633 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.790595    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-podinfo\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.790821 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.790639    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-workload-socket\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.790821 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.790660    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-credential-socket\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.790821 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.790681    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-envoy\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.790821 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.790736    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqjkf\" (UniqueName: \"kubernetes.io/projected/abf68219-6b67-41ae-ab10-ed1a798810f6-kube-api-access-sqjkf\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.790821 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.790763    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-data\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.790821 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.790786    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abf68219-6b67-41ae-ab10-ed1a798810f6-istiod-ca-cert\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.791056 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.790858    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-token\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.791056 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.790897    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-workload-certs\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.892041 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.891999    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-workload-socket\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.892041 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.892042    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-credential-socket\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.892265 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.892073    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-envoy\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.892265 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.892104    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqjkf\" (UniqueName: \"kubernetes.io/projected/abf68219-6b67-41ae-ab10-ed1a798810f6-kube-api-access-sqjkf\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.892265 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.892131    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-data\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.892265 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.892165    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abf68219-6b67-41ae-ab10-ed1a798810f6-istiod-ca-cert\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.892265 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.892239    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-token\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.892570 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.892294    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-workload-certs\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.892570 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.892322    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-podinfo\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.892570 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.892455    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-credential-socket\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.892570 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.892548    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-workload-socket\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.892787 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.892740    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-workload-certs\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.892862 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.892836    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-data\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.893072 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.893046    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abf68219-6b67-41ae-ab10-ed1a798810f6-istiod-ca-cert\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.894449 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.894425    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-envoy\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.894784 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.894767    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-podinfo\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.900149 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.900121    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-token\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:38.900303 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:38.900289    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqjkf\" (UniqueName: \"kubernetes.io/projected/abf68219-6b67-41ae-ab10-ed1a798810f6-kube-api-access-sqjkf\") pod \"e2e-api-b-a335e2-openshift-default-647cf54495-hbm97\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") " pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:39.048204 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:39.048106    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:39.190580 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:39.190555    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"]
Jul 10 04:11:39.192536 ip-10-0-143-142 kubenswrapper[2571]: W0710 04:11:39.192506    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf68219_6b67_41ae_ab10_ed1a798810f6.slice/crio-33c460bf1be433783892028539d0cee42af6e5ab3cebefe10b3d57c52c37a7bb WatchSource:0}: Error finding container 33c460bf1be433783892028539d0cee42af6e5ab3cebefe10b3d57c52c37a7bb: Status 404 returned error can't find the container with id 33c460bf1be433783892028539d0cee42af6e5ab3cebefe10b3d57c52c37a7bb
Jul 10 04:11:39.194530 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:39.194494    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:11:39.194624 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:39.194563    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:11:39.194624 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:39.194594    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:11:39.569360 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:39.569323    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" event={"ID":"abf68219-6b67-41ae-ab10-ed1a798810f6","Type":"ContainerStarted","Data":"a33d7c8c272eefe52c13852aa4752ccfe3b3755763de3fc0863d2ed5bdeeb6ed"}
Jul 10 04:11:39.569593 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:39.569402    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" event={"ID":"abf68219-6b67-41ae-ab10-ed1a798810f6","Type":"ContainerStarted","Data":"33c460bf1be433783892028539d0cee42af6e5ab3cebefe10b3d57c52c37a7bb"}
Jul 10 04:11:39.587983 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:39.587928    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" podStartSLOduration=1.587910452 podStartE2EDuration="1.587910452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:11:39.586314103 +0000 UTC m=+2335.101049412" watchObservedRunningTime="2026-07-10 04:11:39.587910452 +0000 UTC m=+2335.102645755"
Jul 10 04:11:40.049302 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:40.049260    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:40.054528 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:40.054501    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:40.574240 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:40.574200    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:40.575175 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:40.575153    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:56.615693 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:56.615645    2571 patch_prober.go:28] interesting pod/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 04:11:56.616045 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:56.615737    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 04:11:57.684121 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:57.684089    2571 generic.go:356] "Generic (PLEG): container finished" podID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerID="a33d7c8c272eefe52c13852aa4752ccfe3b3755763de3fc0863d2ed5bdeeb6ed" exitCode=0
Jul 10 04:11:57.684547 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:57.684163    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" event={"ID":"abf68219-6b67-41ae-ab10-ed1a798810f6","Type":"ContainerDied","Data":"a33d7c8c272eefe52c13852aa4752ccfe3b3755763de3fc0863d2ed5bdeeb6ed"}
Jul 10 04:11:57.684596 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:57.684554    2571 scope.go:117] "RemoveContainer" containerID="a33d7c8c272eefe52c13852aa4752ccfe3b3755763de3fc0863d2ed5bdeeb6ed"
Jul 10 04:11:57.685532 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:57.685503    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:11:57.685603 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:57.685573    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:11:57.685644 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:57.685605    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:11:58.691995 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:58.691960    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" event={"ID":"abf68219-6b67-41ae-ab10-ed1a798810f6","Type":"ContainerStarted","Data":"452f3b1ff26c46825ff90a67618e6824421fa92d9c858f8f7df9663a0194bfb4"}
Jul 10 04:11:59.048781 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:59.048677    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:11:59.049006 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:59.048977    2571 patch_prober.go:28] interesting pod/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 10 04:11:59.049076 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:11:59.049053    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 10 04:12:00.048726 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:00.048687    2571 patch_prober.go:28] interesting pod/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 10 04:12:00.049168 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:00.048756    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 10 04:12:01.049490 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:01.049441    2571 patch_prober.go:28] interesting pod/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 10 04:12:01.049883 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:01.049522    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 10 04:12:02.048874 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:02.048841    2571 patch_prober.go:28] interesting pod/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 10 04:12:02.049137 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:02.049102    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 10 04:12:03.051807 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:03.051777    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:12:03.052190 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:03.052033    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:12:03.052860 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:03.052842    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:12:46.196915 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:46.196879    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"]
Jul 10 04:12:46.197530 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:46.197323    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerName="istio-proxy" containerID="cri-o://452f3b1ff26c46825ff90a67618e6824421fa92d9c858f8f7df9663a0194bfb4" gracePeriod=30
Jul 10 04:12:48.055152 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:48.055109    2571 patch_prober.go:28] interesting pod/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 04:12:48.055610 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:48.055206    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 04:12:51.452974 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.452949    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:12:51.574179 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574099    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqjkf\" (UniqueName: \"kubernetes.io/projected/abf68219-6b67-41ae-ab10-ed1a798810f6-kube-api-access-sqjkf\") pod \"abf68219-6b67-41ae-ab10-ed1a798810f6\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") "
Jul 10 04:12:51.574179 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574145    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-envoy\") pod \"abf68219-6b67-41ae-ab10-ed1a798810f6\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") "
Jul 10 04:12:51.574179 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574163    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-data\") pod \"abf68219-6b67-41ae-ab10-ed1a798810f6\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") "
Jul 10 04:12:51.574517 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574264    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-podinfo\") pod \"abf68219-6b67-41ae-ab10-ed1a798810f6\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") "
Jul 10 04:12:51.574517 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574389    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-workload-socket\") pod \"abf68219-6b67-41ae-ab10-ed1a798810f6\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") "
Jul 10 04:12:51.574517 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574428    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-credential-socket\") pod \"abf68219-6b67-41ae-ab10-ed1a798810f6\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") "
Jul 10 04:12:51.574674 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574534    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abf68219-6b67-41ae-ab10-ed1a798810f6-istiod-ca-cert\") pod \"abf68219-6b67-41ae-ab10-ed1a798810f6\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") "
Jul 10 04:12:51.574674 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574565    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-token\") pod \"abf68219-6b67-41ae-ab10-ed1a798810f6\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") "
Jul 10 04:12:51.574674 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574590    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-workload-certs\") pod \"abf68219-6b67-41ae-ab10-ed1a798810f6\" (UID: \"abf68219-6b67-41ae-ab10-ed1a798810f6\") "
Jul 10 04:12:51.574911 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574672    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "abf68219-6b67-41ae-ab10-ed1a798810f6" (UID: "abf68219-6b67-41ae-ab10-ed1a798810f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:12:51.574911 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574726    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "abf68219-6b67-41ae-ab10-ed1a798810f6" (UID: "abf68219-6b67-41ae-ab10-ed1a798810f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:12:51.575021 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574918    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "abf68219-6b67-41ae-ab10-ed1a798810f6" (UID: "abf68219-6b67-41ae-ab10-ed1a798810f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:12:51.575021 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574957    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-workload-socket\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:12:51.575021 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.574974    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-credential-socket\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:12:51.575021 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.575008    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abf68219-6b67-41ae-ab10-ed1a798810f6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "abf68219-6b67-41ae-ab10-ed1a798810f6" (UID: "abf68219-6b67-41ae-ab10-ed1a798810f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:12:51.575173 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.575114    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-data" (OuterVolumeSpecName: "istio-data") pod "abf68219-6b67-41ae-ab10-ed1a798810f6" (UID: "abf68219-6b67-41ae-ab10-ed1a798810f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:12:51.576420 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.576394    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "abf68219-6b67-41ae-ab10-ed1a798810f6" (UID: "abf68219-6b67-41ae-ab10-ed1a798810f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:12:51.576568 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.576482    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "abf68219-6b67-41ae-ab10-ed1a798810f6" (UID: "abf68219-6b67-41ae-ab10-ed1a798810f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 04:12:51.576614 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.576565    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf68219-6b67-41ae-ab10-ed1a798810f6-kube-api-access-sqjkf" (OuterVolumeSpecName: "kube-api-access-sqjkf") pod "abf68219-6b67-41ae-ab10-ed1a798810f6" (UID: "abf68219-6b67-41ae-ab10-ed1a798810f6"). InnerVolumeSpecName "kube-api-access-sqjkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:12:51.576822 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.576808    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-token" (OuterVolumeSpecName: "istio-token") pod "abf68219-6b67-41ae-ab10-ed1a798810f6" (UID: "abf68219-6b67-41ae-ab10-ed1a798810f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:12:51.675874 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.675843    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abf68219-6b67-41ae-ab10-ed1a798810f6-istiod-ca-cert\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:12:51.675874 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.675872    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-token\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:12:51.675874 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.675881    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-workload-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:12:51.676101 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.675889    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqjkf\" (UniqueName: \"kubernetes.io/projected/abf68219-6b67-41ae-ab10-ed1a798810f6-kube-api-access-sqjkf\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:12:51.676101 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.675899    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-envoy\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:12:51.676101 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.675906    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-data\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:12:51.676101 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.675913    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abf68219-6b67-41ae-ab10-ed1a798810f6-istio-podinfo\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:12:51.943033 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.942995    2571 generic.go:356] "Generic (PLEG): container finished" podID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerID="452f3b1ff26c46825ff90a67618e6824421fa92d9c858f8f7df9663a0194bfb4" exitCode=0
Jul 10 04:12:51.943198 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.943047    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"
Jul 10 04:12:51.943198 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.943051    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" event={"ID":"abf68219-6b67-41ae-ab10-ed1a798810f6","Type":"ContainerDied","Data":"452f3b1ff26c46825ff90a67618e6824421fa92d9c858f8f7df9663a0194bfb4"}
Jul 10 04:12:51.943198 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.943103    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97" event={"ID":"abf68219-6b67-41ae-ab10-ed1a798810f6","Type":"ContainerDied","Data":"33c460bf1be433783892028539d0cee42af6e5ab3cebefe10b3d57c52c37a7bb"}
Jul 10 04:12:51.943198 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.943128    2571 scope.go:117] "RemoveContainer" containerID="452f3b1ff26c46825ff90a67618e6824421fa92d9c858f8f7df9663a0194bfb4"
Jul 10 04:12:51.952693 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.952668    2571 scope.go:117] "RemoveContainer" containerID="a33d7c8c272eefe52c13852aa4752ccfe3b3755763de3fc0863d2ed5bdeeb6ed"
Jul 10 04:12:51.961008 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.960990    2571 scope.go:117] "RemoveContainer" containerID="452f3b1ff26c46825ff90a67618e6824421fa92d9c858f8f7df9663a0194bfb4"
Jul 10 04:12:51.961309 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:12:51.961291    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"452f3b1ff26c46825ff90a67618e6824421fa92d9c858f8f7df9663a0194bfb4\": container with ID starting with 452f3b1ff26c46825ff90a67618e6824421fa92d9c858f8f7df9663a0194bfb4 not found: ID does not exist" containerID="452f3b1ff26c46825ff90a67618e6824421fa92d9c858f8f7df9663a0194bfb4"
Jul 10 04:12:51.961369 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.961317    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"452f3b1ff26c46825ff90a67618e6824421fa92d9c858f8f7df9663a0194bfb4"} err="failed to get container status \"452f3b1ff26c46825ff90a67618e6824421fa92d9c858f8f7df9663a0194bfb4\": rpc error: code = NotFound desc = could not find container \"452f3b1ff26c46825ff90a67618e6824421fa92d9c858f8f7df9663a0194bfb4\": container with ID starting with 452f3b1ff26c46825ff90a67618e6824421fa92d9c858f8f7df9663a0194bfb4 not found: ID does not exist"
Jul 10 04:12:51.961369 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.961335    2571 scope.go:117] "RemoveContainer" containerID="a33d7c8c272eefe52c13852aa4752ccfe3b3755763de3fc0863d2ed5bdeeb6ed"
Jul 10 04:12:51.961622 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:12:51.961602    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a33d7c8c272eefe52c13852aa4752ccfe3b3755763de3fc0863d2ed5bdeeb6ed\": container with ID starting with a33d7c8c272eefe52c13852aa4752ccfe3b3755763de3fc0863d2ed5bdeeb6ed not found: ID does not exist" containerID="a33d7c8c272eefe52c13852aa4752ccfe3b3755763de3fc0863d2ed5bdeeb6ed"
Jul 10 04:12:51.961693 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.961632    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a33d7c8c272eefe52c13852aa4752ccfe3b3755763de3fc0863d2ed5bdeeb6ed"} err="failed to get container status \"a33d7c8c272eefe52c13852aa4752ccfe3b3755763de3fc0863d2ed5bdeeb6ed\": rpc error: code = NotFound desc = could not find container \"a33d7c8c272eefe52c13852aa4752ccfe3b3755763de3fc0863d2ed5bdeeb6ed\": container with ID starting with a33d7c8c272eefe52c13852aa4752ccfe3b3755763de3fc0863d2ed5bdeeb6ed not found: ID does not exist"
Jul 10 04:12:51.967966 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.967939    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"]
Jul 10 04:12:51.974038 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:51.974012    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a335e2-openshift-default-647cf54495-hbm97"]
Jul 10 04:12:53.126439 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:53.126401    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" path="/var/lib/kubelet/pods/abf68219-6b67-41ae-ab10-ed1a798810f6/volumes"
Jul 10 04:12:56.995173 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:56.995134    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"]
Jul 10 04:12:56.995765 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:56.995748    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerName="istio-proxy"
Jul 10 04:12:56.995765 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:56.995766    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerName="istio-proxy"
Jul 10 04:12:56.995902 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:56.995832    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerName="istio-proxy"
Jul 10 04:12:56.995902 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:56.995841    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerName="istio-proxy"
Jul 10 04:12:57.000283 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.000262    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.002554 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.002525    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-3505cb-openshift-default-dockercfg-6492l\""
Jul 10 04:12:57.012535 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.012509    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"]
Jul 10 04:12:57.124077 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.124018    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-token\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.124077 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.124061    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-podinfo\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.124403 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.124152    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4683051-3d53-4090-8a4d-5a1f30059db2-istiod-ca-cert\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.124403 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.124184    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-credential-socket\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.124403 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.124210    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-data\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.124403 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.124272    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-envoy\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.124403 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.124300    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2blxf\" (UniqueName: \"kubernetes.io/projected/b4683051-3d53-4090-8a4d-5a1f30059db2-kube-api-access-2blxf\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.124403 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.124395    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-workload-socket\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.124693 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.124428    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-workload-certs\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.225377 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.225335    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4683051-3d53-4090-8a4d-5a1f30059db2-istiod-ca-cert\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.225377 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.225383    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-credential-socket\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.225652 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.225407    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-data\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.225652 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.225430    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-envoy\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.225652 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.225453    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2blxf\" (UniqueName: \"kubernetes.io/projected/b4683051-3d53-4090-8a4d-5a1f30059db2-kube-api-access-2blxf\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.225652 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.225526    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-workload-socket\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.225652 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.225548    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-workload-certs\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.225964 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.225938    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-workload-socket\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.226044 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.225938    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-token\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.226044 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.225980    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-workload-certs\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.226044 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.226032    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-podinfo\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.226202 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.226045    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-data\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.226202 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.226159    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4683051-3d53-4090-8a4d-5a1f30059db2-istiod-ca-cert\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.226395 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.226376    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-credential-socket\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.227916 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.227892    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-envoy\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.228362 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.228346    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-podinfo\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.233182 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.233161    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2blxf\" (UniqueName: \"kubernetes.io/projected/b4683051-3d53-4090-8a4d-5a1f30059db2-kube-api-access-2blxf\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.233305 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.233165    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-token\") pod \"e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") " pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.312066 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.311990    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:57.455098 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.455073    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"]
Jul 10 04:12:57.456913 ip-10-0-143-142 kubenswrapper[2571]: W0710 04:12:57.456885    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4683051_3d53_4090_8a4d_5a1f30059db2.slice/crio-7bb5856377612e339635eb7e5b9bedcbbbaf27c2b546ab51a35c8f9371f49b4f WatchSource:0}: Error finding container 7bb5856377612e339635eb7e5b9bedcbbbaf27c2b546ab51a35c8f9371f49b4f: Status 404 returned error can't find the container with id 7bb5856377612e339635eb7e5b9bedcbbbaf27c2b546ab51a35c8f9371f49b4f
Jul 10 04:12:57.459028 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.458991    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:12:57.459107 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.459060    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:12:57.459107 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.459091    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:12:57.974498 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.974442    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" event={"ID":"b4683051-3d53-4090-8a4d-5a1f30059db2","Type":"ContainerStarted","Data":"e8e58bd4e1b2ca9a0d368de18509b23e7cafafb705ace5e2919b36ff5aac0426"}
Jul 10 04:12:57.974687 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:57.974513    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" event={"ID":"b4683051-3d53-4090-8a4d-5a1f30059db2","Type":"ContainerStarted","Data":"7bb5856377612e339635eb7e5b9bedcbbbaf27c2b546ab51a35c8f9371f49b4f"}
Jul 10 04:12:58.001662 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:58.001610    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" podStartSLOduration=2.00159391 podStartE2EDuration="2.00159391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:12:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:12:57.999245702 +0000 UTC m=+2413.513981006" watchObservedRunningTime="2026-07-10 04:12:58.00159391 +0000 UTC m=+2413.516329282"
Jul 10 04:12:58.312766 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:58.312680    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:58.317568 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:58.317544    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:58.979546 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:58.979513    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:12:58.980408 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:12:58.980385    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:13:14.047908 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:14.047806    2571 patch_prober.go:28] interesting pod/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:37580->10.134.0.45:15021: read: connection reset by peer" start-of-body=
Jul 10 04:13:14.047908 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:14.047897    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" podUID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:37580->10.134.0.45:15021: read: connection reset by peer"
Jul 10 04:13:15.063956 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:15.063918    2571 generic.go:356] "Generic (PLEG): container finished" podID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerID="e8e58bd4e1b2ca9a0d368de18509b23e7cafafb705ace5e2919b36ff5aac0426" exitCode=0
Jul 10 04:13:15.064330 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:15.063976    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" event={"ID":"b4683051-3d53-4090-8a4d-5a1f30059db2","Type":"ContainerDied","Data":"e8e58bd4e1b2ca9a0d368de18509b23e7cafafb705ace5e2919b36ff5aac0426"}
Jul 10 04:13:15.064416 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:15.064402    2571 scope.go:117] "RemoveContainer" containerID="e8e58bd4e1b2ca9a0d368de18509b23e7cafafb705ace5e2919b36ff5aac0426"
Jul 10 04:13:15.065471 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:15.065424    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:13:15.065588 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:15.065566    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:13:15.065636 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:15.065614    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 10 04:13:16.072791 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.072758    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" event={"ID":"b4683051-3d53-4090-8a4d-5a1f30059db2","Type":"ContainerStarted","Data":"4b0cdee54a61533e1753cdae144f2e5c6162afe3142949afc8f836b29b284a6f"}
Jul 10 04:13:16.312932 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.312897    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:13:16.313140 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.313110    2571 patch_prober.go:28] interesting pod/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 10 04:13:16.313184 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.313155    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" podUID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 10 04:13:16.398579 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.398493    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"]
Jul 10 04:13:16.399003 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.398985    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerName="istio-proxy"
Jul 10 04:13:16.399073 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.399005    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf68219-6b67-41ae-ab10-ed1a798810f6" containerName="istio-proxy"
Jul 10 04:13:16.401166 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.401143    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.404530 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.404449    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-937cca\"/\"kube-root-ca.crt\""
Jul 10 04:13:16.404660 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.404544    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-937cca\"/\"openshift-service-ca.crt\""
Jul 10 04:13:16.404796 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.404780    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-937cca\"/\"default-dockercfg-vnxxz\""
Jul 10 04:13:16.404867 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.404781    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-937cca\"/\"auth-test-model-937cca-kserve-self-signed-certs\""
Jul 10 04:13:16.415998 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.415973    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"]
Jul 10 04:13:16.509274 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.509225    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-dshm\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.509557 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.509356    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-home\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.509557 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.509402    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a056c77-631a-4c60-9519-44e743b4a66a-tls-certs\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.509557 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.509427    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2qv7\" (UniqueName: \"kubernetes.io/projected/6a056c77-631a-4c60-9519-44e743b4a66a-kube-api-access-h2qv7\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.509557 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.509471    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-model-cache\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.509557 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.509510    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-tmp-dir\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.610797 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.610760    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-home\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.610987 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.610811    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a056c77-631a-4c60-9519-44e743b4a66a-tls-certs\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.610987 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.610845    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2qv7\" (UniqueName: \"kubernetes.io/projected/6a056c77-631a-4c60-9519-44e743b4a66a-kube-api-access-h2qv7\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.610987 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.610876    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-model-cache\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.610987 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.610907    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-tmp-dir\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.610987 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.610970    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-dshm\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.611248 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.611199    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-home\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.611292 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.611262    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-model-cache\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.611328 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.611290    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-tmp-dir\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.613177 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.613146    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a056c77-631a-4c60-9519-44e743b4a66a-tls-certs\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.613311 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.613176    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-dshm\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.620095 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.620066    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2qv7\" (UniqueName: \"kubernetes.io/projected/6a056c77-631a-4c60-9519-44e743b4a66a-kube-api-access-h2qv7\") pod \"auth-test-model-937cca-kserve-7fb46995bc-lrrpd\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") " pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.711174 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.711136    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:16.902071 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:16.902049    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"]
Jul 10 04:13:17.079015 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:17.078983    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd" event={"ID":"6a056c77-631a-4c60-9519-44e743b4a66a","Type":"ContainerStarted","Data":"e175b8439389553c18912f516740145e9f21ca23e33ea9ed46f15c614ea6e1bc"}
Jul 10 04:13:17.079394 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:17.079029    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd" event={"ID":"6a056c77-631a-4c60-9519-44e743b4a66a","Type":"ContainerStarted","Data":"84c9c1f7c40ecb1ab5bab5efba469730d625da3ddf083a0f04da84de52b7dfef"}
Jul 10 04:13:17.095822 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:17.095773    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd" podStartSLOduration=1.095757048 podStartE2EDuration="1.095757048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:13:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:13:17.094447404 +0000 UTC m=+2432.609182706" watchObservedRunningTime="2026-07-10 04:13:17.095757048 +0000 UTC m=+2432.610492353"
Jul 10 04:13:17.313388 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:17.313300    2571 patch_prober.go:28] interesting pod/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 10 04:13:17.313388 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:17.313354    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" podUID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 10 04:13:18.312630 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:18.312594    2571 patch_prober.go:28] interesting pod/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 10 04:13:18.313033 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:18.312653    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" podUID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 10 04:13:19.312732 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:19.312693    2571 patch_prober.go:28] interesting pod/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 10 04:13:19.313128 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:19.312763    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" podUID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 10 04:13:20.316215 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:20.316178    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:13:20.316654 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:20.316582    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:13:20.317267 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:20.317246    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:13:26.711885 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:26.711837    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:26.711885 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:26.711892    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:26.725384 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:26.725356    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:27.136254 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:27.136174    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:13:49.579402 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.579364    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"]
Jul 10 04:13:49.584201 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.584178    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.587218 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.587192    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3505cb\"/\"kube-root-ca.crt\""
Jul 10 04:13:49.587380 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.587249    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3505cb\"/\"auth-test-model-3505cb-kserve-self-signed-certs\""
Jul 10 04:13:49.587380 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.587200    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3505cb\"/\"default-dockercfg-rzvjd\""
Jul 10 04:13:49.587380 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.587200    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3505cb\"/\"openshift-service-ca.crt\""
Jul 10 04:13:49.592972 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.592945    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"]
Jul 10 04:13:49.614709 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.614671    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-tmp-dir\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.614709 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.614710    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hv77\" (UniqueName: \"kubernetes.io/projected/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-kube-api-access-7hv77\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.614917 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.614735    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-home\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.614917 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.614753    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-dshm\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.614917 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.614810    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-tls-certs\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.614917 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.614885    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-model-cache\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.716260 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.716216    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-model-cache\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.716496 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.716326    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-tmp-dir\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.716496 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.716351    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hv77\" (UniqueName: \"kubernetes.io/projected/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-kube-api-access-7hv77\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.716496 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.716385    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-home\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.716679 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.716503    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-dshm\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.716679 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.716540    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-tls-certs\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.716789 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.716719    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-model-cache\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.716789 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.716753    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-tmp-dir\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.716891 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.716835    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-home\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.719001 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.718974    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-dshm\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.719144 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.719127    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-tls-certs\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.724317 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.724289    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hv77\" (UniqueName: \"kubernetes.io/projected/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-kube-api-access-7hv77\") pod \"auth-test-model-3505cb-kserve-678c567c95-m8qvx\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") " pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:49.894139 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:49.894053    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:50.048032 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:50.048002    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"]
Jul 10 04:13:50.050966 ip-10-0-143-142 kubenswrapper[2571]: W0710 04:13:50.050930    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4e79ce9_fa21_4a39_bccd_da6388d4ff04.slice/crio-e9f905751a23cfb38f9587ac08050ddc8de4a7f6638fa03f53229f28b765adb8 WatchSource:0}: Error finding container e9f905751a23cfb38f9587ac08050ddc8de4a7f6638fa03f53229f28b765adb8: Status 404 returned error can't find the container with id e9f905751a23cfb38f9587ac08050ddc8de4a7f6638fa03f53229f28b765adb8
Jul 10 04:13:50.236045 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:50.236000    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx" event={"ID":"c4e79ce9-fa21-4a39-bccd-da6388d4ff04","Type":"ContainerStarted","Data":"da27009df8d2a90a520245e4821741c5e4fc070262026de801a2135546c1bbfc"}
Jul 10 04:13:50.236045 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:50.236052    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx" event={"ID":"c4e79ce9-fa21-4a39-bccd-da6388d4ff04","Type":"ContainerStarted","Data":"e9f905751a23cfb38f9587ac08050ddc8de4a7f6638fa03f53229f28b765adb8"}
Jul 10 04:13:50.251515 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:50.251440    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx" podStartSLOduration=1.2514177530000001 podStartE2EDuration="1.251417753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:13:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:13:50.250643265 +0000 UTC m=+2465.765378569" watchObservedRunningTime="2026-07-10 04:13:50.251417753 +0000 UTC m=+2465.766153062"
Jul 10 04:13:59.894650 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:59.894604    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:59.895046 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:59.894674    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:13:59.907202 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:13:59.907174    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:14:00.294551 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:14:00.294522    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:15:56.279011 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:15:56.278976    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"]
Jul 10 04:15:57.250301 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:15:57.250253    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"]
Jul 10 04:16:07.878816 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:07.878728    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"]
Jul 10 04:16:07.882394 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:07.882365    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:07.884743 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:07.884721    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-937cca\"/\"sub-test-model-937cca-kserve-self-signed-certs\""
Jul 10 04:16:07.893993 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:07.893968    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"]
Jul 10 04:16:08.039910 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.039870    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-home\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.040115 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.039929    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e600dc3-74cc-4cd0-98c3-ead52b12e458-tls-certs\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.040115 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.040026    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-dshm\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.040115 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.040069    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzk5x\" (UniqueName: \"kubernetes.io/projected/2e600dc3-74cc-4cd0-98c3-ead52b12e458-kube-api-access-gzk5x\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.040271 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.040142    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-tmp-dir\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.040271 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.040227    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-model-cache\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.140995 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.140890    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-home\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.140995 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.140941    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e600dc3-74cc-4cd0-98c3-ead52b12e458-tls-certs\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.140995 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.140995    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-dshm\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.141279 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.141032    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzk5x\" (UniqueName: \"kubernetes.io/projected/2e600dc3-74cc-4cd0-98c3-ead52b12e458-kube-api-access-gzk5x\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.141279 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.141082    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-tmp-dir\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.141279 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.141145    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-model-cache\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.141473 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.141337    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-home\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.141473 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.141429    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-tmp-dir\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.141598 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.141549    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-model-cache\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.143315 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.143288    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-dshm\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.143601 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.143580    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e600dc3-74cc-4cd0-98c3-ead52b12e458-tls-certs\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.149102 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.149083    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzk5x\" (UniqueName: \"kubernetes.io/projected/2e600dc3-74cc-4cd0-98c3-ead52b12e458-kube-api-access-gzk5x\") pod \"sub-test-model-937cca-kserve-75d89f5c9f-wfgzp\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") " pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.195199 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.195157    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:08.330869 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.330844    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"]
Jul 10 04:16:08.332946 ip-10-0-143-142 kubenswrapper[2571]: W0710 04:16:08.332910    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e600dc3_74cc_4cd0_98c3_ead52b12e458.slice/crio-a03e6d4db2c60eea7178af2f2bfcde511c614580671cde73595074154723441d WatchSource:0}: Error finding container a03e6d4db2c60eea7178af2f2bfcde511c614580671cde73595074154723441d: Status 404 returned error can't find the container with id a03e6d4db2c60eea7178af2f2bfcde511c614580671cde73595074154723441d
Jul 10 04:16:08.916664 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.916625    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp" event={"ID":"2e600dc3-74cc-4cd0-98c3-ead52b12e458","Type":"ContainerStarted","Data":"e873112acfb3a657553775d43151cfa72e03bef88ea49e5a75008353ec77bf06"}
Jul 10 04:16:08.917122 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.916673    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp" event={"ID":"2e600dc3-74cc-4cd0-98c3-ead52b12e458","Type":"ContainerStarted","Data":"a03e6d4db2c60eea7178af2f2bfcde511c614580671cde73595074154723441d"}
Jul 10 04:16:08.934389 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:08.934336    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp" podStartSLOduration=1.934321527 podStartE2EDuration="1.934321527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:16:08.932097863 +0000 UTC m=+2604.446833166" watchObservedRunningTime="2026-07-10 04:16:08.934321527 +0000 UTC m=+2604.449056847"
Jul 10 04:16:11.299520 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.299449    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd" podUID="6a056c77-631a-4c60-9519-44e743b4a66a" containerName="main" containerID="cri-o://e175b8439389553c18912f516740145e9f21ca23e33ea9ed46f15c614ea6e1bc" gracePeriod=45
Jul 10 04:16:11.560934 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.560870    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:16:11.674021 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.673985    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-model-cache\") pod \"6a056c77-631a-4c60-9519-44e743b4a66a\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") "
Jul 10 04:16:11.674199 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.674085    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-dshm\") pod \"6a056c77-631a-4c60-9519-44e743b4a66a\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") "
Jul 10 04:16:11.674199 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.674102    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-home\") pod \"6a056c77-631a-4c60-9519-44e743b4a66a\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") "
Jul 10 04:16:11.674199 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.674133    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-tmp-dir\") pod \"6a056c77-631a-4c60-9519-44e743b4a66a\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") "
Jul 10 04:16:11.674199 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.674161    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a056c77-631a-4c60-9519-44e743b4a66a-tls-certs\") pod \"6a056c77-631a-4c60-9519-44e743b4a66a\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") "
Jul 10 04:16:11.674199 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.674182    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2qv7\" (UniqueName: \"kubernetes.io/projected/6a056c77-631a-4c60-9519-44e743b4a66a-kube-api-access-h2qv7\") pod \"6a056c77-631a-4c60-9519-44e743b4a66a\" (UID: \"6a056c77-631a-4c60-9519-44e743b4a66a\") "
Jul 10 04:16:11.674513 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.674268    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-model-cache" (OuterVolumeSpecName: "model-cache") pod "6a056c77-631a-4c60-9519-44e743b4a66a" (UID: "6a056c77-631a-4c60-9519-44e743b4a66a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:16:11.674513 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.674416    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6a056c77-631a-4c60-9519-44e743b4a66a" (UID: "6a056c77-631a-4c60-9519-44e743b4a66a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:16:11.674513 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.674435    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-home" (OuterVolumeSpecName: "home") pod "6a056c77-631a-4c60-9519-44e743b4a66a" (UID: "6a056c77-631a-4c60-9519-44e743b4a66a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:16:11.674661 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.674565    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-model-cache\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:16:11.674661 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.674583    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-home\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:16:11.674661 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.674596    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-tmp-dir\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:16:11.676492 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.676454    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-dshm" (OuterVolumeSpecName: "dshm") pod "6a056c77-631a-4c60-9519-44e743b4a66a" (UID: "6a056c77-631a-4c60-9519-44e743b4a66a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:16:11.676492 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.676452    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a056c77-631a-4c60-9519-44e743b4a66a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6a056c77-631a-4c60-9519-44e743b4a66a" (UID: "6a056c77-631a-4c60-9519-44e743b4a66a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:16:11.676617 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.676497    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a056c77-631a-4c60-9519-44e743b4a66a-kube-api-access-h2qv7" (OuterVolumeSpecName: "kube-api-access-h2qv7") pod "6a056c77-631a-4c60-9519-44e743b4a66a" (UID: "6a056c77-631a-4c60-9519-44e743b4a66a"). InnerVolumeSpecName "kube-api-access-h2qv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:16:11.775503 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.775444    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a056c77-631a-4c60-9519-44e743b4a66a-dshm\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:16:11.775503 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.775494    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a056c77-631a-4c60-9519-44e743b4a66a-tls-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:16:11.775503 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.775505    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2qv7\" (UniqueName: \"kubernetes.io/projected/6a056c77-631a-4c60-9519-44e743b4a66a-kube-api-access-h2qv7\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:16:11.934326 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.934289    2571 generic.go:356] "Generic (PLEG): container finished" podID="6a056c77-631a-4c60-9519-44e743b4a66a" containerID="e175b8439389553c18912f516740145e9f21ca23e33ea9ed46f15c614ea6e1bc" exitCode=0
Jul 10 04:16:11.934575 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.934340    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"
Jul 10 04:16:11.934575 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.934346    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd" event={"ID":"6a056c77-631a-4c60-9519-44e743b4a66a","Type":"ContainerDied","Data":"e175b8439389553c18912f516740145e9f21ca23e33ea9ed46f15c614ea6e1bc"}
Jul 10 04:16:11.934575 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.934387    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd" event={"ID":"6a056c77-631a-4c60-9519-44e743b4a66a","Type":"ContainerDied","Data":"84c9c1f7c40ecb1ab5bab5efba469730d625da3ddf083a0f04da84de52b7dfef"}
Jul 10 04:16:11.934575 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.934408    2571 scope.go:117] "RemoveContainer" containerID="e175b8439389553c18912f516740145e9f21ca23e33ea9ed46f15c614ea6e1bc"
Jul 10 04:16:11.943767 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.943748    2571 scope.go:117] "RemoveContainer" containerID="e175b8439389553c18912f516740145e9f21ca23e33ea9ed46f15c614ea6e1bc"
Jul 10 04:16:11.944064 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:16:11.944044    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e175b8439389553c18912f516740145e9f21ca23e33ea9ed46f15c614ea6e1bc\": container with ID starting with e175b8439389553c18912f516740145e9f21ca23e33ea9ed46f15c614ea6e1bc not found: ID does not exist" containerID="e175b8439389553c18912f516740145e9f21ca23e33ea9ed46f15c614ea6e1bc"
Jul 10 04:16:11.944110 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.944074    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e175b8439389553c18912f516740145e9f21ca23e33ea9ed46f15c614ea6e1bc"} err="failed to get container status \"e175b8439389553c18912f516740145e9f21ca23e33ea9ed46f15c614ea6e1bc\": rpc error: code = NotFound desc = could not find container \"e175b8439389553c18912f516740145e9f21ca23e33ea9ed46f15c614ea6e1bc\": container with ID starting with e175b8439389553c18912f516740145e9f21ca23e33ea9ed46f15c614ea6e1bc not found: ID does not exist"
Jul 10 04:16:11.954436 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.954382    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"]
Jul 10 04:16:11.956250 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:11.956226    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/auth-test-model-937cca-kserve-7fb46995bc-lrrpd"]
Jul 10 04:16:12.272773 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.272676    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx" podUID="c4e79ce9-fa21-4a39-bccd-da6388d4ff04" containerName="main" containerID="cri-o://da27009df8d2a90a520245e4821741c5e4fc070262026de801a2135546c1bbfc" gracePeriod=45
Jul 10 04:16:12.534520 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.534435    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:16:12.686305 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.686258    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-tmp-dir\") pod \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") "
Jul 10 04:16:12.686540 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.686398    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-home\") pod \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") "
Jul 10 04:16:12.686540 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.686425    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-tls-certs\") pod \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") "
Jul 10 04:16:12.686540 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.686511    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hv77\" (UniqueName: \"kubernetes.io/projected/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-kube-api-access-7hv77\") pod \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") "
Jul 10 04:16:12.686540 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.686535    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-model-cache\") pod \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") "
Jul 10 04:16:12.686754 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.686567    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-dshm\") pod \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\" (UID: \"c4e79ce9-fa21-4a39-bccd-da6388d4ff04\") "
Jul 10 04:16:12.686754 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.686586    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c4e79ce9-fa21-4a39-bccd-da6388d4ff04" (UID: "c4e79ce9-fa21-4a39-bccd-da6388d4ff04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:16:12.686754 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.686707    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-home" (OuterVolumeSpecName: "home") pod "c4e79ce9-fa21-4a39-bccd-da6388d4ff04" (UID: "c4e79ce9-fa21-4a39-bccd-da6388d4ff04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:16:12.686903 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.686826    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-model-cache" (OuterVolumeSpecName: "model-cache") pod "c4e79ce9-fa21-4a39-bccd-da6388d4ff04" (UID: "c4e79ce9-fa21-4a39-bccd-da6388d4ff04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:16:12.686959 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.686913    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-tmp-dir\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:16:12.686959 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.686935    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-home\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:16:12.688858 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.688835    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-dshm" (OuterVolumeSpecName: "dshm") pod "c4e79ce9-fa21-4a39-bccd-da6388d4ff04" (UID: "c4e79ce9-fa21-4a39-bccd-da6388d4ff04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:16:12.688858 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.688849    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-kube-api-access-7hv77" (OuterVolumeSpecName: "kube-api-access-7hv77") pod "c4e79ce9-fa21-4a39-bccd-da6388d4ff04" (UID: "c4e79ce9-fa21-4a39-bccd-da6388d4ff04"). InnerVolumeSpecName "kube-api-access-7hv77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:16:12.688991 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.688898    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c4e79ce9-fa21-4a39-bccd-da6388d4ff04" (UID: "c4e79ce9-fa21-4a39-bccd-da6388d4ff04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:16:12.787511 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.787395    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-tls-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:16:12.787511 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.787431    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hv77\" (UniqueName: \"kubernetes.io/projected/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-kube-api-access-7hv77\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:16:12.787511 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.787444    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-model-cache\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:16:12.787511 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.787469    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4e79ce9-fa21-4a39-bccd-da6388d4ff04-dshm\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:16:12.940776 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.940743    2571 generic.go:356] "Generic (PLEG): container finished" podID="c4e79ce9-fa21-4a39-bccd-da6388d4ff04" containerID="da27009df8d2a90a520245e4821741c5e4fc070262026de801a2135546c1bbfc" exitCode=0
Jul 10 04:16:12.940949 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.940794    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"
Jul 10 04:16:12.940949 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.940826    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx" event={"ID":"c4e79ce9-fa21-4a39-bccd-da6388d4ff04","Type":"ContainerDied","Data":"da27009df8d2a90a520245e4821741c5e4fc070262026de801a2135546c1bbfc"}
Jul 10 04:16:12.940949 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.940874    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx" event={"ID":"c4e79ce9-fa21-4a39-bccd-da6388d4ff04","Type":"ContainerDied","Data":"e9f905751a23cfb38f9587ac08050ddc8de4a7f6638fa03f53229f28b765adb8"}
Jul 10 04:16:12.940949 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.940897    2571 scope.go:117] "RemoveContainer" containerID="da27009df8d2a90a520245e4821741c5e4fc070262026de801a2135546c1bbfc"
Jul 10 04:16:12.950325 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.950302    2571 scope.go:117] "RemoveContainer" containerID="da27009df8d2a90a520245e4821741c5e4fc070262026de801a2135546c1bbfc"
Jul 10 04:16:12.950668 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:16:12.950646    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da27009df8d2a90a520245e4821741c5e4fc070262026de801a2135546c1bbfc\": container with ID starting with da27009df8d2a90a520245e4821741c5e4fc070262026de801a2135546c1bbfc not found: ID does not exist" containerID="da27009df8d2a90a520245e4821741c5e4fc070262026de801a2135546c1bbfc"
Jul 10 04:16:12.950780 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.950680    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da27009df8d2a90a520245e4821741c5e4fc070262026de801a2135546c1bbfc"} err="failed to get container status \"da27009df8d2a90a520245e4821741c5e4fc070262026de801a2135546c1bbfc\": rpc error: code = NotFound desc = could not find container \"da27009df8d2a90a520245e4821741c5e4fc070262026de801a2135546c1bbfc\": container with ID starting with da27009df8d2a90a520245e4821741c5e4fc070262026de801a2135546c1bbfc not found: ID does not exist"
Jul 10 04:16:12.961024 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.960993    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"]
Jul 10 04:16:12.967283 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:12.967246    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/auth-test-model-3505cb-kserve-678c567c95-m8qvx"]
Jul 10 04:16:13.128243 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:13.128146    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a056c77-631a-4c60-9519-44e743b4a66a" path="/var/lib/kubelet/pods/6a056c77-631a-4c60-9519-44e743b4a66a/volumes"
Jul 10 04:16:13.128704 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:13.128683    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4e79ce9-fa21-4a39-bccd-da6388d4ff04" path="/var/lib/kubelet/pods/c4e79ce9-fa21-4a39-bccd-da6388d4ff04/volumes"
Jul 10 04:16:18.196293 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:18.196250    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:18.196836 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:18.196326    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:18.208781 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:18.208752    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:18.982057 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:18.982028    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:16:44.379195 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.379158    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"]
Jul 10 04:16:44.379669 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.379567    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4e79ce9-fa21-4a39-bccd-da6388d4ff04" containerName="main"
Jul 10 04:16:44.379669 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.379579    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4e79ce9-fa21-4a39-bccd-da6388d4ff04" containerName="main"
Jul 10 04:16:44.379669 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.379591    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a056c77-631a-4c60-9519-44e743b4a66a" containerName="main"
Jul 10 04:16:44.379669 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.379598    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a056c77-631a-4c60-9519-44e743b4a66a" containerName="main"
Jul 10 04:16:44.379669 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.379670    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a056c77-631a-4c60-9519-44e743b4a66a" containerName="main"
Jul 10 04:16:44.379833 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.379685    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4e79ce9-fa21-4a39-bccd-da6388d4ff04" containerName="main"
Jul 10 04:16:44.383088 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.383070    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.385927 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.385905    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3505cb\"/\"openshift-service-ca.crt\""
Jul 10 04:16:44.386031 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.385905    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3505cb\"/\"default-dockercfg-rzvjd\""
Jul 10 04:16:44.386031 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.385906    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3505cb\"/\"kube-root-ca.crt\""
Jul 10 04:16:44.386031 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.385906    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3505cb\"/\"sub-test-model-3505cb-kserve-self-signed-certs\""
Jul 10 04:16:44.391065 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.391036    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"]
Jul 10 04:16:44.484443 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.484405    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-tmp-dir\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.484443 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.484440    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-dshm\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.484698 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.484486    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-tls-certs\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.484698 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.484518    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v2dk\" (UniqueName: \"kubernetes.io/projected/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-kube-api-access-7v2dk\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.484698 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.484549    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-model-cache\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.484698 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.484563    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-home\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.585798 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.585759    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-tmp-dir\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.585798 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.585807    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-dshm\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.586056 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.585838    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-tls-certs\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.586056 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.585865    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v2dk\" (UniqueName: \"kubernetes.io/projected/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-kube-api-access-7v2dk\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.586056 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.585909    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-model-cache\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.586056 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.585931    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-home\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.586294 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.586248    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-tmp-dir\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.586533 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.586510    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-home\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.586697 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.586582    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-model-cache\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.589248 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.589217    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-dshm\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.589402 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.589372    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-tls-certs\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.594356 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.594333    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v2dk\" (UniqueName: \"kubernetes.io/projected/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-kube-api-access-7v2dk\") pod \"sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") " pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.693342 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.693304    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:44.828757 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:44.828731    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"]
Jul 10 04:16:44.831122 ip-10-0-143-142 kubenswrapper[2571]: W0710 04:16:44.831088    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cedb5fa_5759_4ef1_aae6_7be7e7e0228a.slice/crio-ad4e7f2ed54b729a5fa2514df16b69a4c093cca14062967b96550b97bcfa47b0 WatchSource:0}: Error finding container ad4e7f2ed54b729a5fa2514df16b69a4c093cca14062967b96550b97bcfa47b0: Status 404 returned error can't find the container with id ad4e7f2ed54b729a5fa2514df16b69a4c093cca14062967b96550b97bcfa47b0
Jul 10 04:16:45.101250 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:45.101151    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g" event={"ID":"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a","Type":"ContainerStarted","Data":"bfadfcfced6d14074128dff0112dba7b472da643b4428f949aa57716db60bad6"}
Jul 10 04:16:45.101250 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:45.101195    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g" event={"ID":"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a","Type":"ContainerStarted","Data":"ad4e7f2ed54b729a5fa2514df16b69a4c093cca14062967b96550b97bcfa47b0"}
Jul 10 04:16:45.118899 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:45.118837    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g" podStartSLOduration=1.118816507 podStartE2EDuration="1.118816507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:16:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:16:45.116720261 +0000 UTC m=+2640.631455558" watchObservedRunningTime="2026-07-10 04:16:45.118816507 +0000 UTC m=+2640.633551812"
Jul 10 04:16:54.694185 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:54.694141    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:54.694786 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:54.694203    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:54.706751 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:54.706722    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:16:55.164156 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:16:55.164065    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:17:34.202348 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:34.202307    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"]
Jul 10 04:17:35.191161 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:35.191118    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"]
Jul 10 04:17:46.362003 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.361925    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"]
Jul 10 04:17:46.366181 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.366158    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.368359 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.368338    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-937cca\"/\"rate-test-model-937cca-kserve-self-signed-certs\""
Jul 10 04:17:46.370168 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.370144    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-model-cache\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.370296 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.370174    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqrq7\" (UniqueName: \"kubernetes.io/projected/e6551865-fe7b-4b29-a296-39410d35c3f2-kube-api-access-zqrq7\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.370296 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.370204    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6551865-fe7b-4b29-a296-39410d35c3f2-tls-certs\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.370296 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.370248    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-dshm\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.370491 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.370317    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-tmp-dir\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.370491 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.370342    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-home\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.376032 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.376008    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"]
Jul 10 04:17:46.471621 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.471579    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-tmp-dir\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.471621 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.471625    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-home\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.471883 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.471681    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-model-cache\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.471883 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.471700    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqrq7\" (UniqueName: \"kubernetes.io/projected/e6551865-fe7b-4b29-a296-39410d35c3f2-kube-api-access-zqrq7\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.471883 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.471739    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6551865-fe7b-4b29-a296-39410d35c3f2-tls-certs\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.471883 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.471778    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-dshm\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.472109 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.472008    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-tmp-dir\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.472109 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.472052    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-home\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.472109 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.472071    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-model-cache\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.474003 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.473981    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-dshm\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.474166 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.474147    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6551865-fe7b-4b29-a296-39410d35c3f2-tls-certs\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.479675 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.479657    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqrq7\" (UniqueName: \"kubernetes.io/projected/e6551865-fe7b-4b29-a296-39410d35c3f2-kube-api-access-zqrq7\") pod \"rate-test-model-937cca-kserve-7966fdd7d7-2zlkq\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") " pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.676298 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.676258    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:46.814797 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:46.814766    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"]
Jul 10 04:17:46.816600 ip-10-0-143-142 kubenswrapper[2571]: W0710 04:17:46.816572    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6551865_fe7b_4b29_a296_39410d35c3f2.slice/crio-c477f834d43ebca45d69b1b409e34e70adb15dbbda6f4028f3f3c21c61bdf7c8 WatchSource:0}: Error finding container c477f834d43ebca45d69b1b409e34e70adb15dbbda6f4028f3f3c21c61bdf7c8: Status 404 returned error can't find the container with id c477f834d43ebca45d69b1b409e34e70adb15dbbda6f4028f3f3c21c61bdf7c8
Jul 10 04:17:47.405175 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:47.405133    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq" event={"ID":"e6551865-fe7b-4b29-a296-39410d35c3f2","Type":"ContainerStarted","Data":"25f9cfbaccc50799417f0a45dc9e61e599da3d0d19cec0a31f5bb9fa8d284822"}
Jul 10 04:17:47.405175 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:47.405182    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq" event={"ID":"e6551865-fe7b-4b29-a296-39410d35c3f2","Type":"ContainerStarted","Data":"c477f834d43ebca45d69b1b409e34e70adb15dbbda6f4028f3f3c21c61bdf7c8"}
Jul 10 04:17:49.223914 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.223846    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp" podUID="2e600dc3-74cc-4cd0-98c3-ead52b12e458" containerName="main" containerID="cri-o://e873112acfb3a657553775d43151cfa72e03bef88ea49e5a75008353ec77bf06" gracePeriod=45
Jul 10 04:17:49.419625 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.419594    2571 generic.go:356] "Generic (PLEG): container finished" podID="2e600dc3-74cc-4cd0-98c3-ead52b12e458" containerID="e873112acfb3a657553775d43151cfa72e03bef88ea49e5a75008353ec77bf06" exitCode=0
Jul 10 04:17:49.419796 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.419665    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp" event={"ID":"2e600dc3-74cc-4cd0-98c3-ead52b12e458","Type":"ContainerDied","Data":"e873112acfb3a657553775d43151cfa72e03bef88ea49e5a75008353ec77bf06"}
Jul 10 04:17:49.484173 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.484110    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:17:49.499618 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.499585    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-dshm\") pod \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") "
Jul 10 04:17:49.499841 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.499697    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e600dc3-74cc-4cd0-98c3-ead52b12e458-tls-certs\") pod \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") "
Jul 10 04:17:49.499841 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.499727    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzk5x\" (UniqueName: \"kubernetes.io/projected/2e600dc3-74cc-4cd0-98c3-ead52b12e458-kube-api-access-gzk5x\") pod \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") "
Jul 10 04:17:49.499841 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.499775    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-home\") pod \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") "
Jul 10 04:17:49.499841 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.499798    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-model-cache\") pod \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") "
Jul 10 04:17:49.500052 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.499883    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-tmp-dir\") pod \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\" (UID: \"2e600dc3-74cc-4cd0-98c3-ead52b12e458\") "
Jul 10 04:17:49.500507 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.500404    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-home" (OuterVolumeSpecName: "home") pod "2e600dc3-74cc-4cd0-98c3-ead52b12e458" (UID: "2e600dc3-74cc-4cd0-98c3-ead52b12e458"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:17:49.500650 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.500547    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-model-cache" (OuterVolumeSpecName: "model-cache") pod "2e600dc3-74cc-4cd0-98c3-ead52b12e458" (UID: "2e600dc3-74cc-4cd0-98c3-ead52b12e458"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:17:49.500650 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.500574    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2e600dc3-74cc-4cd0-98c3-ead52b12e458" (UID: "2e600dc3-74cc-4cd0-98c3-ead52b12e458"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:17:49.500781 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.500683    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq" podStartSLOduration=3.5006655220000003 podStartE2EDuration="3.500665522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:17:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:17:47.420368622 +0000 UTC m=+2702.935103926" watchObservedRunningTime="2026-07-10 04:17:49.500665522 +0000 UTC m=+2705.015400825"
Jul 10 04:17:49.502244 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.502222    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e600dc3-74cc-4cd0-98c3-ead52b12e458-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2e600dc3-74cc-4cd0-98c3-ead52b12e458" (UID: "2e600dc3-74cc-4cd0-98c3-ead52b12e458"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:17:49.502868 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.502842    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-dshm" (OuterVolumeSpecName: "dshm") pod "2e600dc3-74cc-4cd0-98c3-ead52b12e458" (UID: "2e600dc3-74cc-4cd0-98c3-ead52b12e458"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:17:49.502966 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.502930    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e600dc3-74cc-4cd0-98c3-ead52b12e458-kube-api-access-gzk5x" (OuterVolumeSpecName: "kube-api-access-gzk5x") pod "2e600dc3-74cc-4cd0-98c3-ead52b12e458" (UID: "2e600dc3-74cc-4cd0-98c3-ead52b12e458"). InnerVolumeSpecName "kube-api-access-gzk5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:17:49.601577 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.601542    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-dshm\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:17:49.601577 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.601571    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e600dc3-74cc-4cd0-98c3-ead52b12e458-tls-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:17:49.601577 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.601581    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzk5x\" (UniqueName: \"kubernetes.io/projected/2e600dc3-74cc-4cd0-98c3-ead52b12e458-kube-api-access-gzk5x\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:17:49.601815 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.601591    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-home\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:17:49.601815 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.601600    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-model-cache\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:17:49.601815 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:49.601609    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e600dc3-74cc-4cd0-98c3-ead52b12e458-tmp-dir\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:17:50.211992 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.211955    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g" podUID="0cedb5fa-5759-4ef1-aae6-7be7e7e0228a" containerName="main" containerID="cri-o://bfadfcfced6d14074128dff0112dba7b472da643b4428f949aa57716db60bad6" gracePeriod=45
Jul 10 04:17:50.427080 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.427049    2571 generic.go:356] "Generic (PLEG): container finished" podID="0cedb5fa-5759-4ef1-aae6-7be7e7e0228a" containerID="bfadfcfced6d14074128dff0112dba7b472da643b4428f949aa57716db60bad6" exitCode=0
Jul 10 04:17:50.427543 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.427119    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g" event={"ID":"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a","Type":"ContainerDied","Data":"bfadfcfced6d14074128dff0112dba7b472da643b4428f949aa57716db60bad6"}
Jul 10 04:17:50.428489 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.428444    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp" event={"ID":"2e600dc3-74cc-4cd0-98c3-ead52b12e458","Type":"ContainerDied","Data":"a03e6d4db2c60eea7178af2f2bfcde511c614580671cde73595074154723441d"}
Jul 10 04:17:50.428489 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.428487    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"
Jul 10 04:17:50.428636 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.428514    2571 scope.go:117] "RemoveContainer" containerID="e873112acfb3a657553775d43151cfa72e03bef88ea49e5a75008353ec77bf06"
Jul 10 04:17:50.451666 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.451641    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"]
Jul 10 04:17:50.456989 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.456965    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/sub-test-model-937cca-kserve-75d89f5c9f-wfgzp"]
Jul 10 04:17:50.470351 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.470291    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:17:50.511903 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.511869    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-model-cache\") pod \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") "
Jul 10 04:17:50.512101 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.511932    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-home\") pod \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") "
Jul 10 04:17:50.512101 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.511992    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-dshm\") pod \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") "
Jul 10 04:17:50.512101 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.512049    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-tls-certs\") pod \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") "
Jul 10 04:17:50.512101 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.512092    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-tmp-dir\") pod \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") "
Jul 10 04:17:50.512318 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.512151    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7v2dk\" (UniqueName: \"kubernetes.io/projected/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-kube-api-access-7v2dk\") pod \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\" (UID: \"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a\") "
Jul 10 04:17:50.512318 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.512228    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-model-cache" (OuterVolumeSpecName: "model-cache") pod "0cedb5fa-5759-4ef1-aae6-7be7e7e0228a" (UID: "0cedb5fa-5759-4ef1-aae6-7be7e7e0228a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:17:50.512318 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.512247    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-home" (OuterVolumeSpecName: "home") pod "0cedb5fa-5759-4ef1-aae6-7be7e7e0228a" (UID: "0cedb5fa-5759-4ef1-aae6-7be7e7e0228a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:17:50.512556 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.512369    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0cedb5fa-5759-4ef1-aae6-7be7e7e0228a" (UID: "0cedb5fa-5759-4ef1-aae6-7be7e7e0228a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:17:50.512601 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.512575    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-tmp-dir\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:17:50.512601 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.512595    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-model-cache\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:17:50.512678 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.512606    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-home\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:17:50.514220 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.514194    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-kube-api-access-7v2dk" (OuterVolumeSpecName: "kube-api-access-7v2dk") pod "0cedb5fa-5759-4ef1-aae6-7be7e7e0228a" (UID: "0cedb5fa-5759-4ef1-aae6-7be7e7e0228a"). InnerVolumeSpecName "kube-api-access-7v2dk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:17:50.514331 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.514296    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-dshm" (OuterVolumeSpecName: "dshm") pod "0cedb5fa-5759-4ef1-aae6-7be7e7e0228a" (UID: "0cedb5fa-5759-4ef1-aae6-7be7e7e0228a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:17:50.514513 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.514487    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0cedb5fa-5759-4ef1-aae6-7be7e7e0228a" (UID: "0cedb5fa-5759-4ef1-aae6-7be7e7e0228a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:17:50.613814 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.613776    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-dshm\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:17:50.613814 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.613806    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-tls-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:17:50.613814 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:50.613817    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v2dk\" (UniqueName: \"kubernetes.io/projected/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a-kube-api-access-7v2dk\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:17:51.128830 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:51.128798    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e600dc3-74cc-4cd0-98c3-ead52b12e458" path="/var/lib/kubelet/pods/2e600dc3-74cc-4cd0-98c3-ead52b12e458/volumes"
Jul 10 04:17:51.435112 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:51.435078    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"
Jul 10 04:17:51.435112 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:51.435099    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g" event={"ID":"0cedb5fa-5759-4ef1-aae6-7be7e7e0228a","Type":"ContainerDied","Data":"ad4e7f2ed54b729a5fa2514df16b69a4c093cca14062967b96550b97bcfa47b0"}
Jul 10 04:17:51.435768 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:51.435153    2571 scope.go:117] "RemoveContainer" containerID="bfadfcfced6d14074128dff0112dba7b472da643b4428f949aa57716db60bad6"
Jul 10 04:17:51.450705 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:51.450677    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"]
Jul 10 04:17:51.453997 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:51.453969    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/sub-test-model-3505cb-kserve-578ff5d8b7-w2m6g"]
Jul 10 04:17:53.128182 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:53.128152    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cedb5fa-5759-4ef1-aae6-7be7e7e0228a" path="/var/lib/kubelet/pods/0cedb5fa-5759-4ef1-aae6-7be7e7e0228a/volumes"
Jul 10 04:17:56.676817 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:56.676785    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:56.677272 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:56.676838    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:56.689451 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:56.689423    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:17:57.477180 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:17:57.477151    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:18:13.370448 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.370411    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"]
Jul 10 04:18:13.370982 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.370963    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cedb5fa-5759-4ef1-aae6-7be7e7e0228a" containerName="main"
Jul 10 04:18:13.371064 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.370985    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cedb5fa-5759-4ef1-aae6-7be7e7e0228a" containerName="main"
Jul 10 04:18:13.371064 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.371020    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e600dc3-74cc-4cd0-98c3-ead52b12e458" containerName="main"
Jul 10 04:18:13.371064 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.371029    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e600dc3-74cc-4cd0-98c3-ead52b12e458" containerName="main"
Jul 10 04:18:13.371304 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.371117    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cedb5fa-5759-4ef1-aae6-7be7e7e0228a" containerName="main"
Jul 10 04:18:13.371304 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.371135    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e600dc3-74cc-4cd0-98c3-ead52b12e458" containerName="main"
Jul 10 04:18:13.373369 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.373347    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.376554 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.376533    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3505cb\"/\"kube-root-ca.crt\""
Jul 10 04:18:13.376554 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.376540    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3505cb\"/\"rate-test-model-3505cb-kserve-self-signed-certs\""
Jul 10 04:18:13.376723 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.376533    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3505cb\"/\"default-dockercfg-rzvjd\""
Jul 10 04:18:13.376723 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.376543    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3505cb\"/\"openshift-service-ca.crt\""
Jul 10 04:18:13.381691 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.381672    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"]
Jul 10 04:18:13.533115 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.533078    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-model-cache\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.533286 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.533147    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-home\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.533286 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.533185    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-dshm\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.533286 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.533207    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-tmp-dir\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.533383 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.533282    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55tnw\" (UniqueName: \"kubernetes.io/projected/725fa53e-3447-4563-ab04-b5a211593c91-kube-api-access-55tnw\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.533383 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.533347    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/725fa53e-3447-4563-ab04-b5a211593c91-tls-certs\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.634702 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.634616    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-model-cache\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.634702 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.634685    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-home\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.634909 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.634727    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-dshm\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.634909 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.634761    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-tmp-dir\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.634909 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.634796    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55tnw\" (UniqueName: \"kubernetes.io/projected/725fa53e-3447-4563-ab04-b5a211593c91-kube-api-access-55tnw\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.634909 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.634831    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/725fa53e-3447-4563-ab04-b5a211593c91-tls-certs\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.635118 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.635029    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-home\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.635118 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.635089    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-model-cache\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.635230 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.635205    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-tmp-dir\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.637118 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.637097    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-dshm\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.637300 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.637283    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/725fa53e-3447-4563-ab04-b5a211593c91-tls-certs\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.641639 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.641620    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55tnw\" (UniqueName: \"kubernetes.io/projected/725fa53e-3447-4563-ab04-b5a211593c91-kube-api-access-55tnw\") pod \"rate-test-model-3505cb-kserve-5fb67845c9-w6nqv\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") " pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.684065 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.684037    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:13.822652 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:13.822624    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"]
Jul 10 04:18:13.824512 ip-10-0-143-142 kubenswrapper[2571]: W0710 04:18:13.824482    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod725fa53e_3447_4563_ab04_b5a211593c91.slice/crio-b0ec9dad66759d7e2f26646b3023b9f9e5a59d18f7ab9e3b9f6c4a2dafe2069c WatchSource:0}: Error finding container b0ec9dad66759d7e2f26646b3023b9f9e5a59d18f7ab9e3b9f6c4a2dafe2069c: Status 404 returned error can't find the container with id b0ec9dad66759d7e2f26646b3023b9f9e5a59d18f7ab9e3b9f6c4a2dafe2069c
Jul 10 04:18:14.559114 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:14.559070    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv" event={"ID":"725fa53e-3447-4563-ab04-b5a211593c91","Type":"ContainerStarted","Data":"82e7a101b7e3db5bcd1247cc5021ee5a872e47c14eda9b7a4dd8ac3587e616c9"}
Jul 10 04:18:14.559114 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:14.559117    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv" event={"ID":"725fa53e-3447-4563-ab04-b5a211593c91","Type":"ContainerStarted","Data":"b0ec9dad66759d7e2f26646b3023b9f9e5a59d18f7ab9e3b9f6c4a2dafe2069c"}
Jul 10 04:18:14.575078 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:14.575016    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv" podStartSLOduration=1.5749946590000001 podStartE2EDuration="1.574994659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 04:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:18:14.573373475 +0000 UTC m=+2730.088108777" watchObservedRunningTime="2026-07-10 04:18:14.574994659 +0000 UTC m=+2730.089729965"
Jul 10 04:18:23.684229 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:23.684189    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:23.684647 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:23.684250    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:23.696656 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:23.696625    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:18:24.617034 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:18:24.617008    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:19:07.100129 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:07.100045    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"]
Jul 10 04:19:08.050922 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:08.050882    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"]
Jul 10 04:19:22.120255 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.120192    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq" podUID="e6551865-fe7b-4b29-a296-39410d35c3f2" containerName="main" containerID="cri-o://25f9cfbaccc50799417f0a45dc9e61e599da3d0d19cec0a31f5bb9fa8d284822" gracePeriod=45
Jul 10 04:19:22.378821 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.378755    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:19:22.467605 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.467572    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6551865-fe7b-4b29-a296-39410d35c3f2-tls-certs\") pod \"e6551865-fe7b-4b29-a296-39410d35c3f2\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") "
Jul 10 04:19:22.467780 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.467638    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-tmp-dir\") pod \"e6551865-fe7b-4b29-a296-39410d35c3f2\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") "
Jul 10 04:19:22.467780 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.467756    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-model-cache\") pod \"e6551865-fe7b-4b29-a296-39410d35c3f2\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") "
Jul 10 04:19:22.467887 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.467789    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-home\") pod \"e6551865-fe7b-4b29-a296-39410d35c3f2\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") "
Jul 10 04:19:22.467887 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.467816    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-dshm\") pod \"e6551865-fe7b-4b29-a296-39410d35c3f2\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") "
Jul 10 04:19:22.467887 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.467844    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqrq7\" (UniqueName: \"kubernetes.io/projected/e6551865-fe7b-4b29-a296-39410d35c3f2-kube-api-access-zqrq7\") pod \"e6551865-fe7b-4b29-a296-39410d35c3f2\" (UID: \"e6551865-fe7b-4b29-a296-39410d35c3f2\") "
Jul 10 04:19:22.468047 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.467880    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e6551865-fe7b-4b29-a296-39410d35c3f2" (UID: "e6551865-fe7b-4b29-a296-39410d35c3f2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:19:22.468107 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.468039    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-model-cache" (OuterVolumeSpecName: "model-cache") pod "e6551865-fe7b-4b29-a296-39410d35c3f2" (UID: "e6551865-fe7b-4b29-a296-39410d35c3f2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:19:22.468107 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.468062    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-home" (OuterVolumeSpecName: "home") pod "e6551865-fe7b-4b29-a296-39410d35c3f2" (UID: "e6551865-fe7b-4b29-a296-39410d35c3f2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:19:22.468206 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.468146    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-tmp-dir\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:19:22.468206 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.468164    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-model-cache\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:19:22.468206 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.468177    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-home\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:19:22.469786 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.469757    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6551865-fe7b-4b29-a296-39410d35c3f2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e6551865-fe7b-4b29-a296-39410d35c3f2" (UID: "e6551865-fe7b-4b29-a296-39410d35c3f2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:19:22.469903 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.469822    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-dshm" (OuterVolumeSpecName: "dshm") pod "e6551865-fe7b-4b29-a296-39410d35c3f2" (UID: "e6551865-fe7b-4b29-a296-39410d35c3f2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:19:22.469903 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.469845    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6551865-fe7b-4b29-a296-39410d35c3f2-kube-api-access-zqrq7" (OuterVolumeSpecName: "kube-api-access-zqrq7") pod "e6551865-fe7b-4b29-a296-39410d35c3f2" (UID: "e6551865-fe7b-4b29-a296-39410d35c3f2"). InnerVolumeSpecName "kube-api-access-zqrq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:19:22.569104 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.569071    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6551865-fe7b-4b29-a296-39410d35c3f2-dshm\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:19:22.569104 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.569099    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqrq7\" (UniqueName: \"kubernetes.io/projected/e6551865-fe7b-4b29-a296-39410d35c3f2-kube-api-access-zqrq7\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:19:22.569104 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.569109    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6551865-fe7b-4b29-a296-39410d35c3f2-tls-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:19:22.884786 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.884750    2571 generic.go:356] "Generic (PLEG): container finished" podID="e6551865-fe7b-4b29-a296-39410d35c3f2" containerID="25f9cfbaccc50799417f0a45dc9e61e599da3d0d19cec0a31f5bb9fa8d284822" exitCode=0
Jul 10 04:19:22.884930 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.884796    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"
Jul 10 04:19:22.884930 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.884810    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq" event={"ID":"e6551865-fe7b-4b29-a296-39410d35c3f2","Type":"ContainerDied","Data":"25f9cfbaccc50799417f0a45dc9e61e599da3d0d19cec0a31f5bb9fa8d284822"}
Jul 10 04:19:22.884930 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.884850    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq" event={"ID":"e6551865-fe7b-4b29-a296-39410d35c3f2","Type":"ContainerDied","Data":"c477f834d43ebca45d69b1b409e34e70adb15dbbda6f4028f3f3c21c61bdf7c8"}
Jul 10 04:19:22.884930 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.884872    2571 scope.go:117] "RemoveContainer" containerID="25f9cfbaccc50799417f0a45dc9e61e599da3d0d19cec0a31f5bb9fa8d284822"
Jul 10 04:19:22.893762 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.893745    2571 scope.go:117] "RemoveContainer" containerID="25f9cfbaccc50799417f0a45dc9e61e599da3d0d19cec0a31f5bb9fa8d284822"
Jul 10 04:19:22.894037 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:19:22.894015    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25f9cfbaccc50799417f0a45dc9e61e599da3d0d19cec0a31f5bb9fa8d284822\": container with ID starting with 25f9cfbaccc50799417f0a45dc9e61e599da3d0d19cec0a31f5bb9fa8d284822 not found: ID does not exist" containerID="25f9cfbaccc50799417f0a45dc9e61e599da3d0d19cec0a31f5bb9fa8d284822"
Jul 10 04:19:22.894088 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.894046    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25f9cfbaccc50799417f0a45dc9e61e599da3d0d19cec0a31f5bb9fa8d284822"} err="failed to get container status \"25f9cfbaccc50799417f0a45dc9e61e599da3d0d19cec0a31f5bb9fa8d284822\": rpc error: code = NotFound desc = could not find container \"25f9cfbaccc50799417f0a45dc9e61e599da3d0d19cec0a31f5bb9fa8d284822\": container with ID starting with 25f9cfbaccc50799417f0a45dc9e61e599da3d0d19cec0a31f5bb9fa8d284822 not found: ID does not exist"
Jul 10 04:19:22.904135 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.904104    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"]
Jul 10 04:19:22.906304 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:22.906284    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-937cca/rate-test-model-937cca-kserve-7966fdd7d7-2zlkq"]
Jul 10 04:19:23.071884 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.071848    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv" podUID="725fa53e-3447-4563-ab04-b5a211593c91" containerName="main" containerID="cri-o://82e7a101b7e3db5bcd1247cc5021ee5a872e47c14eda9b7a4dd8ac3587e616c9" gracePeriod=45
Jul 10 04:19:23.127244 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.127208    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6551865-fe7b-4b29-a296-39410d35c3f2" path="/var/lib/kubelet/pods/e6551865-fe7b-4b29-a296-39410d35c3f2/volumes"
Jul 10 04:19:23.321260 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.321232    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:19:23.376256 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.376221    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55tnw\" (UniqueName: \"kubernetes.io/projected/725fa53e-3447-4563-ab04-b5a211593c91-kube-api-access-55tnw\") pod \"725fa53e-3447-4563-ab04-b5a211593c91\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") "
Jul 10 04:19:23.376449 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.376308    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/725fa53e-3447-4563-ab04-b5a211593c91-tls-certs\") pod \"725fa53e-3447-4563-ab04-b5a211593c91\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") "
Jul 10 04:19:23.376449 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.376361    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-model-cache\") pod \"725fa53e-3447-4563-ab04-b5a211593c91\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") "
Jul 10 04:19:23.376449 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.376384    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-home\") pod \"725fa53e-3447-4563-ab04-b5a211593c91\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") "
Jul 10 04:19:23.376665 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.376506    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-tmp-dir\") pod \"725fa53e-3447-4563-ab04-b5a211593c91\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") "
Jul 10 04:19:23.376665 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.376605    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-dshm\") pod \"725fa53e-3447-4563-ab04-b5a211593c91\" (UID: \"725fa53e-3447-4563-ab04-b5a211593c91\") "
Jul 10 04:19:23.376665 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.376627    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-model-cache" (OuterVolumeSpecName: "model-cache") pod "725fa53e-3447-4563-ab04-b5a211593c91" (UID: "725fa53e-3447-4563-ab04-b5a211593c91"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:19:23.376665 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.376645    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-home" (OuterVolumeSpecName: "home") pod "725fa53e-3447-4563-ab04-b5a211593c91" (UID: "725fa53e-3447-4563-ab04-b5a211593c91"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:19:23.376829 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.376763    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "725fa53e-3447-4563-ab04-b5a211593c91" (UID: "725fa53e-3447-4563-ab04-b5a211593c91"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:19:23.376949 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.376936    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-model-cache\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:19:23.376991 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.376952    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-home\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:19:23.376991 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.376965    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-tmp-dir\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:19:23.378383 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.378356    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/725fa53e-3447-4563-ab04-b5a211593c91-kube-api-access-55tnw" (OuterVolumeSpecName: "kube-api-access-55tnw") pod "725fa53e-3447-4563-ab04-b5a211593c91" (UID: "725fa53e-3447-4563-ab04-b5a211593c91"). InnerVolumeSpecName "kube-api-access-55tnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:19:23.378500 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.378399    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/725fa53e-3447-4563-ab04-b5a211593c91-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "725fa53e-3447-4563-ab04-b5a211593c91" (UID: "725fa53e-3447-4563-ab04-b5a211593c91"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 04:19:23.378500 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.378440    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-dshm" (OuterVolumeSpecName: "dshm") pod "725fa53e-3447-4563-ab04-b5a211593c91" (UID: "725fa53e-3447-4563-ab04-b5a211593c91"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:19:23.477956 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.477917    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/725fa53e-3447-4563-ab04-b5a211593c91-dshm\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:19:23.477956 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.477950    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55tnw\" (UniqueName: \"kubernetes.io/projected/725fa53e-3447-4563-ab04-b5a211593c91-kube-api-access-55tnw\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:19:23.477956 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.477961    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/725fa53e-3447-4563-ab04-b5a211593c91-tls-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:19:23.890765 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.890676    2571 generic.go:356] "Generic (PLEG): container finished" podID="725fa53e-3447-4563-ab04-b5a211593c91" containerID="82e7a101b7e3db5bcd1247cc5021ee5a872e47c14eda9b7a4dd8ac3587e616c9" exitCode=0
Jul 10 04:19:23.890765 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.890725    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"
Jul 10 04:19:23.890983 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.890762    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv" event={"ID":"725fa53e-3447-4563-ab04-b5a211593c91","Type":"ContainerDied","Data":"82e7a101b7e3db5bcd1247cc5021ee5a872e47c14eda9b7a4dd8ac3587e616c9"}
Jul 10 04:19:23.890983 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.890810    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv" event={"ID":"725fa53e-3447-4563-ab04-b5a211593c91","Type":"ContainerDied","Data":"b0ec9dad66759d7e2f26646b3023b9f9e5a59d18f7ab9e3b9f6c4a2dafe2069c"}
Jul 10 04:19:23.890983 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.890833    2571 scope.go:117] "RemoveContainer" containerID="82e7a101b7e3db5bcd1247cc5021ee5a872e47c14eda9b7a4dd8ac3587e616c9"
Jul 10 04:19:23.899619 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.899603    2571 scope.go:117] "RemoveContainer" containerID="82e7a101b7e3db5bcd1247cc5021ee5a872e47c14eda9b7a4dd8ac3587e616c9"
Jul 10 04:19:23.899863 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:19:23.899847    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82e7a101b7e3db5bcd1247cc5021ee5a872e47c14eda9b7a4dd8ac3587e616c9\": container with ID starting with 82e7a101b7e3db5bcd1247cc5021ee5a872e47c14eda9b7a4dd8ac3587e616c9 not found: ID does not exist" containerID="82e7a101b7e3db5bcd1247cc5021ee5a872e47c14eda9b7a4dd8ac3587e616c9"
Jul 10 04:19:23.899901 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.899870    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82e7a101b7e3db5bcd1247cc5021ee5a872e47c14eda9b7a4dd8ac3587e616c9"} err="failed to get container status \"82e7a101b7e3db5bcd1247cc5021ee5a872e47c14eda9b7a4dd8ac3587e616c9\": rpc error: code = NotFound desc = could not find container \"82e7a101b7e3db5bcd1247cc5021ee5a872e47c14eda9b7a4dd8ac3587e616c9\": container with ID starting with 82e7a101b7e3db5bcd1247cc5021ee5a872e47c14eda9b7a4dd8ac3587e616c9 not found: ID does not exist"
Jul 10 04:19:23.909361 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.909336    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"]
Jul 10 04:19:23.913262 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:23.913240    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3505cb/rate-test-model-3505cb-kserve-5fb67845c9-w6nqv"]
Jul 10 04:19:25.126231 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:19:25.126195    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="725fa53e-3447-4563-ab04-b5a211593c91" path="/var/lib/kubelet/pods/725fa53e-3447-4563-ab04-b5a211593c91/volumes"
Jul 10 04:20:18.773454 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:18.773421    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"]
Jul 10 04:20:18.774018 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:18.773705    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" podUID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerName="istio-proxy" containerID="cri-o://4b0cdee54a61533e1753cdae144f2e5c6162afe3142949afc8f836b29b284a6f" gracePeriod=30
Jul 10 04:20:20.318552 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:20.318508    2571 patch_prober.go:28] interesting pod/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 04:20:20.318936 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:20.318582    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" podUID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 04:20:24.029808 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.029777    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:20:24.118773 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.118699    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4683051-3d53-4090-8a4d-5a1f30059db2-istiod-ca-cert\") pod \"b4683051-3d53-4090-8a4d-5a1f30059db2\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") "
Jul 10 04:20:24.118773 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.118755    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-token\") pod \"b4683051-3d53-4090-8a4d-5a1f30059db2\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") "
Jul 10 04:20:24.119007 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.118795    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-workload-socket\") pod \"b4683051-3d53-4090-8a4d-5a1f30059db2\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") "
Jul 10 04:20:24.119007 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.118828    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-workload-certs\") pod \"b4683051-3d53-4090-8a4d-5a1f30059db2\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") "
Jul 10 04:20:24.119007 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.118899    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2blxf\" (UniqueName: \"kubernetes.io/projected/b4683051-3d53-4090-8a4d-5a1f30059db2-kube-api-access-2blxf\") pod \"b4683051-3d53-4090-8a4d-5a1f30059db2\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") "
Jul 10 04:20:24.119007 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.118982    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-podinfo\") pod \"b4683051-3d53-4090-8a4d-5a1f30059db2\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") "
Jul 10 04:20:24.119211 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.119008    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-credential-socket\") pod \"b4683051-3d53-4090-8a4d-5a1f30059db2\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") "
Jul 10 04:20:24.119211 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.119033    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-envoy\") pod \"b4683051-3d53-4090-8a4d-5a1f30059db2\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") "
Jul 10 04:20:24.119211 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.119064    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-data\") pod \"b4683051-3d53-4090-8a4d-5a1f30059db2\" (UID: \"b4683051-3d53-4090-8a4d-5a1f30059db2\") "
Jul 10 04:20:24.119211 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.119184    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b4683051-3d53-4090-8a4d-5a1f30059db2" (UID: "b4683051-3d53-4090-8a4d-5a1f30059db2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:20:24.119488 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.119259    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4683051-3d53-4090-8a4d-5a1f30059db2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b4683051-3d53-4090-8a4d-5a1f30059db2" (UID: "b4683051-3d53-4090-8a4d-5a1f30059db2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 04:20:24.119488 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.119312    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b4683051-3d53-4090-8a4d-5a1f30059db2" (UID: "b4683051-3d53-4090-8a4d-5a1f30059db2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:20:24.119488 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.119409    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4683051-3d53-4090-8a4d-5a1f30059db2-istiod-ca-cert\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:20:24.119488 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.119415    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b4683051-3d53-4090-8a4d-5a1f30059db2" (UID: "b4683051-3d53-4090-8a4d-5a1f30059db2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:20:24.119488 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.119426    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-workload-socket\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:20:24.119984 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.119966    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-data" (OuterVolumeSpecName: "istio-data") pod "b4683051-3d53-4090-8a4d-5a1f30059db2" (UID: "b4683051-3d53-4090-8a4d-5a1f30059db2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:20:24.121212 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.121181    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4683051-3d53-4090-8a4d-5a1f30059db2" (UID: "b4683051-3d53-4090-8a4d-5a1f30059db2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:20:24.121360 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.121208    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4683051-3d53-4090-8a4d-5a1f30059db2-kube-api-access-2blxf" (OuterVolumeSpecName: "kube-api-access-2blxf") pod "b4683051-3d53-4090-8a4d-5a1f30059db2" (UID: "b4683051-3d53-4090-8a4d-5a1f30059db2"). InnerVolumeSpecName "kube-api-access-2blxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 04:20:24.121649 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.121628    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b4683051-3d53-4090-8a4d-5a1f30059db2" (UID: "b4683051-3d53-4090-8a4d-5a1f30059db2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 04:20:24.121707 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.121630    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b4683051-3d53-4090-8a4d-5a1f30059db2" (UID: "b4683051-3d53-4090-8a4d-5a1f30059db2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 04:20:24.182261 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.182229    2571 generic.go:356] "Generic (PLEG): container finished" podID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerID="4b0cdee54a61533e1753cdae144f2e5c6162afe3142949afc8f836b29b284a6f" exitCode=0
Jul 10 04:20:24.182452 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.182290    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"
Jul 10 04:20:24.182452 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.182312    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" event={"ID":"b4683051-3d53-4090-8a4d-5a1f30059db2","Type":"ContainerDied","Data":"4b0cdee54a61533e1753cdae144f2e5c6162afe3142949afc8f836b29b284a6f"}
Jul 10 04:20:24.182452 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.182369    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz" event={"ID":"b4683051-3d53-4090-8a4d-5a1f30059db2","Type":"ContainerDied","Data":"7bb5856377612e339635eb7e5b9bedcbbbaf27c2b546ab51a35c8f9371f49b4f"}
Jul 10 04:20:24.182452 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.182393    2571 scope.go:117] "RemoveContainer" containerID="4b0cdee54a61533e1753cdae144f2e5c6162afe3142949afc8f836b29b284a6f"
Jul 10 04:20:24.191340 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.191325    2571 scope.go:117] "RemoveContainer" containerID="e8e58bd4e1b2ca9a0d368de18509b23e7cafafb705ace5e2919b36ff5aac0426"
Jul 10 04:20:24.199358 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.199338    2571 scope.go:117] "RemoveContainer" containerID="4b0cdee54a61533e1753cdae144f2e5c6162afe3142949afc8f836b29b284a6f"
Jul 10 04:20:24.199637 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:20:24.199614    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b0cdee54a61533e1753cdae144f2e5c6162afe3142949afc8f836b29b284a6f\": container with ID starting with 4b0cdee54a61533e1753cdae144f2e5c6162afe3142949afc8f836b29b284a6f not found: ID does not exist" containerID="4b0cdee54a61533e1753cdae144f2e5c6162afe3142949afc8f836b29b284a6f"
Jul 10 04:20:24.199728 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.199648    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b0cdee54a61533e1753cdae144f2e5c6162afe3142949afc8f836b29b284a6f"} err="failed to get container status \"4b0cdee54a61533e1753cdae144f2e5c6162afe3142949afc8f836b29b284a6f\": rpc error: code = NotFound desc = could not find container \"4b0cdee54a61533e1753cdae144f2e5c6162afe3142949afc8f836b29b284a6f\": container with ID starting with 4b0cdee54a61533e1753cdae144f2e5c6162afe3142949afc8f836b29b284a6f not found: ID does not exist"
Jul 10 04:20:24.199728 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.199671    2571 scope.go:117] "RemoveContainer" containerID="e8e58bd4e1b2ca9a0d368de18509b23e7cafafb705ace5e2919b36ff5aac0426"
Jul 10 04:20:24.199911 ip-10-0-143-142 kubenswrapper[2571]: E0710 04:20:24.199892    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8e58bd4e1b2ca9a0d368de18509b23e7cafafb705ace5e2919b36ff5aac0426\": container with ID starting with e8e58bd4e1b2ca9a0d368de18509b23e7cafafb705ace5e2919b36ff5aac0426 not found: ID does not exist" containerID="e8e58bd4e1b2ca9a0d368de18509b23e7cafafb705ace5e2919b36ff5aac0426"
Jul 10 04:20:24.199954 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.199920    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8e58bd4e1b2ca9a0d368de18509b23e7cafafb705ace5e2919b36ff5aac0426"} err="failed to get container status \"e8e58bd4e1b2ca9a0d368de18509b23e7cafafb705ace5e2919b36ff5aac0426\": rpc error: code = NotFound desc = could not find container \"e8e58bd4e1b2ca9a0d368de18509b23e7cafafb705ace5e2919b36ff5aac0426\": container with ID starting with e8e58bd4e1b2ca9a0d368de18509b23e7cafafb705ace5e2919b36ff5aac0426 not found: ID does not exist"
Jul 10 04:20:24.205637 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.205616    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"]
Jul 10 04:20:24.208933 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.208914    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-3505cb-openshift-default-54cbfff895-vnssz"]
Jul 10 04:20:24.220396 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.220372    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2blxf\" (UniqueName: \"kubernetes.io/projected/b4683051-3d53-4090-8a4d-5a1f30059db2-kube-api-access-2blxf\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:20:24.220487 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.220395    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-podinfo\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:20:24.220487 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.220406    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-credential-socket\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:20:24.220487 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.220414    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-envoy\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:20:24.220487 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.220423    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-data\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:20:24.220487 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.220431    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4683051-3d53-4090-8a4d-5a1f30059db2-istio-token\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:20:24.220487 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.220438    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4683051-3d53-4090-8a4d-5a1f30059db2-workload-certs\") on node \"ip-10-0-143-142.ec2.internal\" DevicePath \"\""
Jul 10 04:20:24.504452 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.504418    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-g4zvw_008f3b93-2fa3-4193-9eb5-c49b118edf92/manager/0.log"
Jul 10 04:20:24.629984 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.629939    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9qwfg_51022770-1212-4d2f-abed-3853fc50624c/manager/0.log"
Jul 10 04:20:24.886294 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:24.886190    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-sxbh5_d549e6f8-9589-4a94-a8e3-49d44eb3276c/server/0.log"
Jul 10 04:20:25.009135 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:25.009101    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mg5lt_6052a68a-6878-4bec-ab1d-27c9acc6f346/manager/0.log"
Jul 10 04:20:25.126016 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:25.125986    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4683051-3d53-4090-8a4d-5a1f30059db2" path="/var/lib/kubelet/pods/b4683051-3d53-4090-8a4d-5a1f30059db2/volumes"
Jul 10 04:20:25.400491 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:25.400433    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d676bbf4b-j4p7p_0c25b42d-cf89-4ee2-945a-44c91292719e/manager/0.log"
Jul 10 04:20:25.414715 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:25.414690    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d676bbf4b-j4p7p_0c25b42d-cf89-4ee2-945a-44c91292719e/copy-manifests/0.log"
Jul 10 04:20:25.793881 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:25.793854    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6dbbf545f9-mdtmj_b4ae5589-365b-4565-b0a4-ba128bc8297e/maas-api/0.log"
Jul 10 04:20:27.608414 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:27.608373    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-4s256_eceb4843-a35d-4eaf-aa03-2459830a6811/postgres/0.log"
Jul 10 04:20:29.118265 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:29.118238    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-b2ntp_8885b43a-19d0-4f65-9f22-0b8f3b5aba6a/manager/0.log"
Jul 10 04:20:29.233925 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:29.233891    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hl7p9_cb887640-90c8-4e82-8b16-b91764e68298/manager/0.log"
Jul 10 04:20:29.472598 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:29.472568    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-57rgw_84bf255d-1361-42a7-91da-eda7f9af1c97/registry-server/0.log"
Jul 10 04:20:29.598877 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:29.598849    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-msdr4_a837360a-4250-446f-bc24-d71f7a422884/manager/0.log"
Jul 10 04:20:29.839191 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:29.839109    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vlshf_b00144cf-d55c-4961-8607-d76ed6f17139/manager/0.log"
Jul 10 04:20:30.207272 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:30.207224    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4599pm_6e0815c6-dc6d-4a74-8621-fbe550124c15/istio-proxy/0.log"
Jul 10 04:20:30.444837 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:30.444809    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d9645bfdd-2grjn_4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a/kube-auth-proxy/0.log"
Jul 10 04:20:30.688611 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:30.688568    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-d5244_736e2dc2-32c4-4d9a-ba85-c9122ce188a0/istio-proxy/1.log"
Jul 10 04:20:31.532790 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:31.532764    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-hf6fh_fd3f1056-33b8-40ec-8c38-cbc930677fa4/main/0.log"
Jul 10 04:20:31.646757 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:31.646717    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-zkjfs_5f69d2bf-5584-41c3-a513-65b127f066c6/main/0.log"
Jul 10 04:20:32.001178 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:32.001151    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-md567_c625d684-5d06-49e5-a901-356978c2b090/main/0.log"
Jul 10 04:20:38.695842 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:38.695739    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vlr5v_859d3328-63da-49c4-a871-2b45a4960764/global-pull-secret-syncer/0.log"
Jul 10 04:20:38.815171 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:38.815137    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8458m_8c21afc9-23db-4b36-b482-8c244a254df6/konnectivity-agent/0.log"
Jul 10 04:20:38.882961 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:38.882912    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-142.ec2.internal_0b3b8930aa1846297e408b672c61a6ab/haproxy/0.log"
Jul 10 04:20:43.831401 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:43.831369    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-b2ntp_8885b43a-19d0-4f65-9f22-0b8f3b5aba6a/manager/0.log"
Jul 10 04:20:43.860686 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:43.860661    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hl7p9_cb887640-90c8-4e82-8b16-b91764e68298/manager/0.log"
Jul 10 04:20:43.927167 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:43.927122    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-57rgw_84bf255d-1361-42a7-91da-eda7f9af1c97/registry-server/0.log"
Jul 10 04:20:44.050493 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:44.050442    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-msdr4_a837360a-4250-446f-bc24-d71f7a422884/manager/0.log"
Jul 10 04:20:44.176099 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:44.176065    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vlshf_b00144cf-d55c-4961-8607-d76ed6f17139/manager/0.log"
Jul 10 04:20:46.232377 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:46.232348    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rxbc8_3f1d8a47-c493-483c-a550-935c74a11a07/node-exporter/0.log"
Jul 10 04:20:46.253829 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:46.253805    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rxbc8_3f1d8a47-c493-483c-a550-935c74a11a07/kube-rbac-proxy/0.log"
Jul 10 04:20:46.273664 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:46.273638    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rxbc8_3f1d8a47-c493-483c-a550-935c74a11a07/init-textfile/0.log"
Jul 10 04:20:46.732662 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:46.732630    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d49776db7-gtg5m_4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453/telemeter-client/0.log"
Jul 10 04:20:46.756203 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:46.756176    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d49776db7-gtg5m_4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453/reload/0.log"
Jul 10 04:20:46.776179 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:46.776148    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d49776db7-gtg5m_4fcc25bf-dcbc-4fd1-bfcc-84c0fae13453/kube-rbac-proxy/0.log"
Jul 10 04:20:49.372936 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.372899    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"]
Jul 10 04:20:49.373315 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.373299    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerName="istio-proxy"
Jul 10 04:20:49.373373 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.373316    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerName="istio-proxy"
Jul 10 04:20:49.373373 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.373329    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerName="istio-proxy"
Jul 10 04:20:49.373373 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.373338    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerName="istio-proxy"
Jul 10 04:20:49.373483 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.373380    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="725fa53e-3447-4563-ab04-b5a211593c91" containerName="main"
Jul 10 04:20:49.373483 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.373390    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="725fa53e-3447-4563-ab04-b5a211593c91" containerName="main"
Jul 10 04:20:49.373483 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.373406    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6551865-fe7b-4b29-a296-39410d35c3f2" containerName="main"
Jul 10 04:20:49.373483 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.373414    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6551865-fe7b-4b29-a296-39410d35c3f2" containerName="main"
Jul 10 04:20:49.373611 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.373508    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4683051-3d53-4090-8a4d-5a1f30059db2" containerName="istio-proxy"
Jul 10 04:20:49.373611 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.373523    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="725fa53e-3447-4563-ab04-b5a211593c91" containerName="main"
Jul 10 04:20:49.373611 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.373534    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6551865-fe7b-4b29-a296-39410d35c3f2" containerName="main"
Jul 10 04:20:49.378412 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.378389    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.380847 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.380825    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pg24g\"/\"openshift-service-ca.crt\""
Jul 10 04:20:49.381622 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.381604    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pg24g\"/\"kube-root-ca.crt\""
Jul 10 04:20:49.381622 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.381613    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pg24g\"/\"default-dockercfg-96jld\""
Jul 10 04:20:49.385558 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.385525    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"]
Jul 10 04:20:49.547064 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.547020    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f6923c25-f12c-4429-bee4-c7693d9164fb-podres\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.547064 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.547067    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6923c25-f12c-4429-bee4-c7693d9164fb-sys\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.547309 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.547124    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f6923c25-f12c-4429-bee4-c7693d9164fb-proc\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.547309 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.547145    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbk7s\" (UniqueName: \"kubernetes.io/projected/f6923c25-f12c-4429-bee4-c7693d9164fb-kube-api-access-xbk7s\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.547309 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.547217    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6923c25-f12c-4429-bee4-c7693d9164fb-lib-modules\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.648267 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.648166    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f6923c25-f12c-4429-bee4-c7693d9164fb-podres\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.648267 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.648211    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6923c25-f12c-4429-bee4-c7693d9164fb-sys\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.648267 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.648257    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f6923c25-f12c-4429-bee4-c7693d9164fb-proc\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.648604 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.648282    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbk7s\" (UniqueName: \"kubernetes.io/projected/f6923c25-f12c-4429-bee4-c7693d9164fb-kube-api-access-xbk7s\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.648604 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.648355    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6923c25-f12c-4429-bee4-c7693d9164fb-lib-modules\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.648604 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.648380    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f6923c25-f12c-4429-bee4-c7693d9164fb-proc\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.648604 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.648383    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6923c25-f12c-4429-bee4-c7693d9164fb-sys\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.648604 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.648356    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f6923c25-f12c-4429-bee4-c7693d9164fb-podres\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.648604 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.648505    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6923c25-f12c-4429-bee4-c7693d9164fb-lib-modules\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.656110 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.656084    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbk7s\" (UniqueName: \"kubernetes.io/projected/f6923c25-f12c-4429-bee4-c7693d9164fb-kube-api-access-xbk7s\") pod \"perf-node-gather-daemonset-qq98m\" (UID: \"f6923c25-f12c-4429-bee4-c7693d9164fb\") " pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.688954 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.688916    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:49.826519 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:49.826490    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"]
Jul 10 04:20:49.828364 ip-10-0-143-142 kubenswrapper[2571]: W0710 04:20:49.828334    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf6923c25_f12c_4429_bee4_c7693d9164fb.slice/crio-0afbb1aa5a051ef8f0ca26192ac9bfe2c65424c0b7762876e441910634ea2fa2 WatchSource:0}: Error finding container 0afbb1aa5a051ef8f0ca26192ac9bfe2c65424c0b7762876e441910634ea2fa2: Status 404 returned error can't find the container with id 0afbb1aa5a051ef8f0ca26192ac9bfe2c65424c0b7762876e441910634ea2fa2
Jul 10 04:20:50.305168 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:50.305075    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m" event={"ID":"f6923c25-f12c-4429-bee4-c7693d9164fb","Type":"ContainerStarted","Data":"014259cd3df21431b098f58794be176012175b6dd73b1ffc802b19508cf959bb"}
Jul 10 04:20:50.305168 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:50.305132    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m" event={"ID":"f6923c25-f12c-4429-bee4-c7693d9164fb","Type":"ContainerStarted","Data":"0afbb1aa5a051ef8f0ca26192ac9bfe2c65424c0b7762876e441910634ea2fa2"}
Jul 10 04:20:50.305355 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:50.305199    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:50.320763 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:50.320713    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m" podStartSLOduration=1.320695977 podStartE2EDuration="1.320695977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 04:20:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 04:20:50.31858086 +0000 UTC m=+2885.833316162" watchObservedRunningTime="2026-07-10 04:20:50.320695977 +0000 UTC m=+2885.835431280"
Jul 10 04:20:51.069898 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:51.069866    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wszvx_a3a9c521-10bc-4303-beee-302cee1be142/dns/0.log"
Jul 10 04:20:51.089909 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:51.089881    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wszvx_a3a9c521-10bc-4303-beee-302cee1be142/kube-rbac-proxy/0.log"
Jul 10 04:20:51.110774 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:51.110744    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9p7cz_4b1d7593-1d0c-4166-8dc8-733dc6c4f55c/dns-node-resolver/0.log"
Jul 10 04:20:51.649086 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:51.649051    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-667654d5f7-qvstj_a0518043-17c4-426e-9bc7-3a49f827589f/registry/0.log"
Jul 10 04:20:51.713069 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:51.713035    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xnfdx_bfe8cc34-69bf-4d9e-a945-b1c867b6111a/node-ca/0.log"
Jul 10 04:20:52.621005 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:52.620975    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4599pm_6e0815c6-dc6d-4a74-8621-fbe550124c15/istio-proxy/0.log"
Jul 10 04:20:52.753003 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:52.752966    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d9645bfdd-2grjn_4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a/kube-auth-proxy/0.log"
Jul 10 04:20:52.802856 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:52.802827    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-d5244_736e2dc2-32c4-4d9a-ba85-c9122ce188a0/istio-proxy/0.log"
Jul 10 04:20:52.830845 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:52.830816    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-d5244_736e2dc2-32c4-4d9a-ba85-c9122ce188a0/istio-proxy/1.log"
Jul 10 04:20:53.579758 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:53.579728    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hdnl8_86afb721-e30b-48ed-b932-7ceca0182679/serve-healthcheck-canary/0.log"
Jul 10 04:20:54.089473 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:54.089430    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqmg8_9df437ac-228c-4177-84cd-effed0e43caf/kube-rbac-proxy/0.log"
Jul 10 04:20:54.112969 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:54.112944    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqmg8_9df437ac-228c-4177-84cd-effed0e43caf/exporter/0.log"
Jul 10 04:20:54.133483 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:54.133439    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqmg8_9df437ac-228c-4177-84cd-effed0e43caf/extractor/0.log"
Jul 10 04:20:56.320736 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:56.320707    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pg24g/perf-node-gather-daemonset-qq98m"
Jul 10 04:20:56.355932 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:56.355905    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-g4zvw_008f3b93-2fa3-4193-9eb5-c49b118edf92/manager/0.log"
Jul 10 04:20:56.486575 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:56.486543    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9qwfg_51022770-1212-4d2f-abed-3853fc50624c/manager/0.log"
Jul 10 04:20:56.631081 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:56.630993    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-sxbh5_d549e6f8-9589-4a94-a8e3-49d44eb3276c/server/0.log"
Jul 10 04:20:56.697490 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:56.697436    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mg5lt_6052a68a-6878-4bec-ab1d-27c9acc6f346/manager/0.log"
Jul 10 04:20:56.853026 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:56.852993    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d676bbf4b-j4p7p_0c25b42d-cf89-4ee2-945a-44c91292719e/manager/0.log"
Jul 10 04:20:56.869571 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:56.869546    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d676bbf4b-j4p7p_0c25b42d-cf89-4ee2-945a-44c91292719e/copy-manifests/0.log"
Jul 10 04:20:59.225387 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:59.225347    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4599pm_6e0815c6-dc6d-4a74-8621-fbe550124c15/istio-proxy/0.log"
Jul 10 04:20:59.341913 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:59.341881    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d9645bfdd-2grjn_4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a/kube-auth-proxy/0.log"
Jul 10 04:20:59.374007 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:59.373975    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-d5244_736e2dc2-32c4-4d9a-ba85-c9122ce188a0/istio-proxy/0.log"
Jul 10 04:20:59.402397 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:20:59.402370    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-d5244_736e2dc2-32c4-4d9a-ba85-c9122ce188a0/istio-proxy/1.log"
Jul 10 04:21:00.675995 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:00.675966    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4599pm_6e0815c6-dc6d-4a74-8621-fbe550124c15/istio-proxy/0.log"
Jul 10 04:21:00.795195 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:00.795165    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d9645bfdd-2grjn_4bcc8dc3-bac1-49c8-b038-d2bebfe5df3a/kube-auth-proxy/0.log"
Jul 10 04:21:00.826817 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:00.826787    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-d5244_736e2dc2-32c4-4d9a-ba85-c9122ce188a0/istio-proxy/0.log"
Jul 10 04:21:00.854703 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:00.854673    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-d5244_736e2dc2-32c4-4d9a-ba85-c9122ce188a0/istio-proxy/1.log"
Jul 10 04:21:04.751195 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:04.751163    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7dqr4_638ec913-d0a6-4937-bc25-adc4d6fc5897/kube-multus/0.log"
Jul 10 04:21:05.067422 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:05.067339    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dhvb5_26fcd562-4c93-4255-9a6d-0c4a362b4b72/kube-multus-additional-cni-plugins/0.log"
Jul 10 04:21:05.087297 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:05.087269    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dhvb5_26fcd562-4c93-4255-9a6d-0c4a362b4b72/egress-router-binary-copy/0.log"
Jul 10 04:21:05.106142 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:05.106117    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dhvb5_26fcd562-4c93-4255-9a6d-0c4a362b4b72/cni-plugins/0.log"
Jul 10 04:21:05.125926 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:05.125890    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dhvb5_26fcd562-4c93-4255-9a6d-0c4a362b4b72/bond-cni-plugin/0.log"
Jul 10 04:21:05.145569 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:05.145540    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dhvb5_26fcd562-4c93-4255-9a6d-0c4a362b4b72/routeoverride-cni/0.log"
Jul 10 04:21:05.165389 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:05.165362    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dhvb5_26fcd562-4c93-4255-9a6d-0c4a362b4b72/whereabouts-cni-bincopy/0.log"
Jul 10 04:21:05.184766 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:05.184738    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dhvb5_26fcd562-4c93-4255-9a6d-0c4a362b4b72/whereabouts-cni/0.log"
Jul 10 04:21:05.330005 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:05.329930    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j6vpn_26778894-1f27-4bcc-b3de-5ae0ac3aca4c/network-metrics-daemon/0.log"
Jul 10 04:21:05.348064 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:05.348040    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j6vpn_26778894-1f27-4bcc-b3de-5ae0ac3aca4c/kube-rbac-proxy/0.log"
Jul 10 04:21:06.252664 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:06.252637    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gxvf2_d4200887-8985-4033-8fb6-5660c89ff1f7/ovn-controller/0.log"
Jul 10 04:21:06.269301 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:06.269275    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gxvf2_d4200887-8985-4033-8fb6-5660c89ff1f7/ovn-acl-logging/0.log"
Jul 10 04:21:06.291964 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:06.291940    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gxvf2_d4200887-8985-4033-8fb6-5660c89ff1f7/kube-rbac-proxy-node/0.log"
Jul 10 04:21:06.311596 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:06.311568    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gxvf2_d4200887-8985-4033-8fb6-5660c89ff1f7/kube-rbac-proxy-ovn-metrics/0.log"
Jul 10 04:21:06.328127 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:06.328101    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gxvf2_d4200887-8985-4033-8fb6-5660c89ff1f7/northd/0.log"
Jul 10 04:21:06.348911 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:06.348887    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gxvf2_d4200887-8985-4033-8fb6-5660c89ff1f7/nbdb/0.log"
Jul 10 04:21:06.368913 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:06.368887    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gxvf2_d4200887-8985-4033-8fb6-5660c89ff1f7/sbdb/0.log"
Jul 10 04:21:06.419415 ip-10-0-143-142 kubenswrapper[2571]: I0710 04:21:06.419385    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gxvf2_d4200887-8985-4033-8fb6-5660c89ff1f7/ovnkube-controller/0.log"