Sep 08 15:07:09.383134 ip-10-0-132-76 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 08 15:07:09.383143 ip-10-0-132-76 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 08 15:07:09.383152 ip-10-0-132-76 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 08 15:07:09.383357 ip-10-0-132-76 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 08 15:07:19.412617 ip-10-0-132-76 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 08 15:07:19.412654 ip-10-0-132-76 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 3c9af281352f4f6298750ec5d7d76ce7 --
Sep 08 15:09:10.730320 ip-10-0-132-76 systemd[1]: Starting Kubernetes Kubelet...
Sep 08 15:09:11.107911 ip-10-0-132-76 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 15:09:11.107911 ip-10-0-132-76 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 08 15:09:11.107911 ip-10-0-132-76 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 15:09:11.107911 ip-10-0-132-76 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 08 15:09:11.107911 ip-10-0-132-76 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 15:09:11.108549 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.107969    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 08 15:09:11.110985 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.110970    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:09:11.110985 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.110985    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.110989    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.110992    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.110995    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.110999    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111004    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111008    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111011    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111016    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111019    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111022    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111025    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111027    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111030    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111045    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111048    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111051    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111054    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111056    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:09:11.111059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111059    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111062    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111064    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111068    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111072    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111076    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111079    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111082    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111084    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111087    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111090    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111092    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111095    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111098    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111100    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111103    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111106    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111108    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111111    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:09:11.111507 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111113    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111115    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111118    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111121    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111123    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111125    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111128    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111132    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111134    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111136    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111139    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111141    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111144    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111146    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111149    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111152    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111155    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111157    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111160    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111163    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:09:11.111960 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111165    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111167    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111170    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111173    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111175    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111178    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111181    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111183    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111186    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111189    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111198    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111201    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111203    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111206    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111208    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111211    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111213    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111216    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111218    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111221    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:09:11.112455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111223    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111225    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111228    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111230    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111233    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111235    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111238    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111241    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111243    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111246    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111249    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111251    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111254    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111256    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111259    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111261    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111264    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111266    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111269    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.111272    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:09:11.113014 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112023    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112030    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112046    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112057    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112061    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112064    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112067    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112069    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112072    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112075    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112077    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112080    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112082    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112085    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112087    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112090    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112093    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112095    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112097    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112101    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:09:11.113516 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112106    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112109    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112112    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112115    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112118    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112121    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112124    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112126    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112129    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112132    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112134    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112137    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112139    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112142    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112144    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112146    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112154    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112157    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112160    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112162    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:09:11.114022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112165    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112168    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112170    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112173    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112178    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112181    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112184    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112187    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112190    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112192    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112195    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112199    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112202    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112205    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112207    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112210    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112212    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112215    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112217    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:09:11.114536 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112220    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112222    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112225    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112227    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112230    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112232    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112235    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112237    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112240    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112242    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112250    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112253    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112255    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112258    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112260    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112263    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112266    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112268    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112271    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112273    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:09:11.115001 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112276    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112278    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112281    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112283    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112288    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112290    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112292    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112295    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112297    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112300    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112304    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112307    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112309    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112312    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112315    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112317    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112320    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112322    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112324    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112327    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:09:11.115521 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112688    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112694    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112698    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112701    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112703    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112706    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112709    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112712    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112714    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112717    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112720    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112722    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112725    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112727    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112731    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112735    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112740    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112743    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112746    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112749    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:09:11.116002 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112753    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112756    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112759    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112761    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112764    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112768    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112772    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112775    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112777    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112780    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112782    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112785    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112787    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112790    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112794    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112797    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112800    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112803    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112806    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:09:11.116499 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112809    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112812    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112815    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112817    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112820    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112822    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112825    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112827    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112830    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112834    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112836    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112839    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112841    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112844    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112846    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112849    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112851    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112854    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112856    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112859    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:09:11.117003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112861    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112864    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112866    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112869    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112871    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112874    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112876    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112879    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112882    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112885    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112888    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112890    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112893    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112895    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112898    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112900    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112902    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112905    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112908    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112910    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:09:11.117495 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112913    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112917    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112920    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112922    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112925    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112927    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112930    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112932    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112935    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112938    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112940    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112943    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112945    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112948    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112950    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112952    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112955    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112957    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112960    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.112963    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:09:11.118011 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113028    2575 flags.go:64] FLAG: --address="0.0.0.0"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113089    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113096    2575 flags.go:64] FLAG: --anonymous-auth="true"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113101    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113105    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113109    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113113    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113117    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113120    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113123    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113126    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113130    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113133    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113138    2575 flags.go:64] FLAG: --cgroup-root=""
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113141    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113144    2575 flags.go:64] FLAG: --client-ca-file=""
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113147    2575 flags.go:64] FLAG: --cloud-provider="external"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113150    2575 flags.go:64] FLAG: --cluster-dns="[]"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113155    2575 flags.go:64] FLAG: --cluster-domain=""
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113158    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113161    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113164    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113167    2575 flags.go:64] FLAG: --container-log-max-files="5"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113171    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 08 15:09:11.118503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113174    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113177    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113180    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113184    2575 flags.go:64] FLAG: --contention-profiling="false"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113187    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113189    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113193    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113196    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113199    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113202    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113206    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113209    2575 flags.go:64] FLAG: --enable-load-reader="false"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113213    2575 flags.go:64] FLAG: --enable-server="true"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113216    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113220    2575 flags.go:64] FLAG: --event-burst="100"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113223    2575 flags.go:64] FLAG: --event-qps="50"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113226    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113229    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113232    2575 flags.go:64] FLAG: --eviction-hard=""
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113236    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113239    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113243    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113246    2575 flags.go:64] FLAG: --eviction-soft=""
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113249    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113252    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 08 15:09:11.119090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113255    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113258    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113261    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113264    2575 flags.go:64] FLAG: --fail-swap-on="true"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113267    2575 flags.go:64] FLAG: --feature-gates=""
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113270    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113273    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113276    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113280    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113283    2575 flags.go:64] FLAG: --healthz-port="10248"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113286    2575 flags.go:64] FLAG: --help="false"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113289    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113292    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113295    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113298    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113301    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113304    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113307    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113310    2575 flags.go:64] FLAG: --image-service-endpoint=""
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113314    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113317    2575 flags.go:64] FLAG: --kube-api-burst="100"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113320    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113323    2575 flags.go:64] FLAG: --kube-api-qps="50"
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113326    2575 flags.go:64] FLAG: --kube-reserved=""
Sep 08 15:09:11.119716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113329    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113331    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113335    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113338    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113342    2575 flags.go:64] FLAG: --lock-file=""
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113345    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113348    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113351    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113356    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113359    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113362    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113364    2575 flags.go:64] FLAG: --logging-format="text"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113367    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113371    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113373    2575 flags.go:64] FLAG: --manifest-url=""
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113376    2575 flags.go:64] FLAG: --manifest-url-header=""
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113380    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113383    2575 flags.go:64] FLAG: --max-open-files="1000000"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113387    2575 flags.go:64] FLAG: --max-pods="110"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113390    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113393    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113396    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113399    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113402    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113405    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 08 15:09:11.120348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113408    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113415    2575 flags.go:64] FLAG: --node-status-max-images="50"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113418    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113422    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113425    2575 flags.go:64] FLAG: --pod-cidr=""
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113428    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113433    2575 flags.go:64] FLAG: --pod-manifest-path=""
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113435    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113439    2575 flags.go:64] FLAG: --pods-per-core="0"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113441    2575 flags.go:64] FLAG: --port="10250"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113445    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113448    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a7955a0c13283a73"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113452    2575 flags.go:64] FLAG: --qos-reserved=""
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113455    2575 flags.go:64] FLAG: --read-only-port="10255"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113458    2575 flags.go:64] FLAG: --register-node="true"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113461    2575 flags.go:64] FLAG: --register-with-taints=""
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113465    2575 flags.go:64] FLAG: --registry-burst="10"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113467    2575 flags.go:64] FLAG: --registry-qps="5"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113470    2575 flags.go:64] FLAG: --reserved-cpus=""
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113473    2575 flags.go:64] FLAG: --reserved-memory=""
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113476    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113480    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113482    2575 flags.go:64] FLAG: --rotate-certificates="false"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113485    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113488    2575 flags.go:64] FLAG: --runonce="false"
Sep 08 15:09:11.121213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113491    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113494    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113497    2575 flags.go:64] FLAG: --seccomp-default="false"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113500    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113502    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113505    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113508    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113511    2575 flags.go:64] FLAG: --storage-driver-password="root"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113514    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113517    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113520    2575 flags.go:64] FLAG: --storage-driver-user="root"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113523    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113526    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113529    2575 flags.go:64] FLAG: --system-cgroups=""
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113532    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113538    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113540    2575 flags.go:64] FLAG: --tls-cert-file=""
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113543    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113549    2575 flags.go:64] FLAG: --tls-min-version=""
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113553    2575 flags.go:64] FLAG: --tls-private-key-file=""
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113556    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113559    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113562    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113565    2575 flags.go:64] FLAG: --v="2"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113569    2575 flags.go:64] FLAG: --version="false"
Sep 08 15:09:11.122085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113572    2575 flags.go:64] FLAG: --vmodule=""
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113576    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.113580    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113687    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113691    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113695    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113700    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113703    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113707    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113709    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113712    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113715    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113717    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113720    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113723    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113725    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113728    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113730    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113733    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:09:11.122692 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113736    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113739    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113741    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113744    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113747    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113749    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113753    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113757    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113759    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113762    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113764    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113767    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113770    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113772    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113775    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113777    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113780    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113783    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113785    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113788    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:09:11.123170 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113791    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113793    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113797    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113801    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113803    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113807    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113810    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113812    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113815    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113818    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113821    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113824    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113826    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113830    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113833    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113836    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113838    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113841    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113844    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113849    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:09:11.123660 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113852    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113854    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113857    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113860    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113862    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113865    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113867    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113870    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113872    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113875    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113878    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113881    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113883    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113886    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113888    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113890    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113893    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113896    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113898    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113901    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:09:11.124162 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113903    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113906    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113908    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113911    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113913    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113916    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113919    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113921    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113924    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113926    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113930    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113933    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113936    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113938    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113941    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113944    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113946    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113949    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113952    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113954    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:09:11.124652 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113957    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113960    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.113962    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.114452    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.122655    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.122670    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.123973    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.123986    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124066    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124073    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124077    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124080    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124083    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124086    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:09:11.125164 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124089    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124092    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124095    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124098    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124101    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124104    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124107    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124110    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124112    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124115    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124119    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124121    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124124    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124127    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124129    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124132    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124135    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124138    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124140    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124143    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:09:11.125551 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124146    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124149    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124151    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124155    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124158    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124160    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124163    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124166    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124169    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124172    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124174    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124177    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124180    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124182    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124185    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124187    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124190    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124198    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124201    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124203    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:09:11.126148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124207    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124210    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124212    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124215    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124217    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124220    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124223    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124225    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124228    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124232    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124235    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124238    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124241    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124244    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124247    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124251    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124255    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124258    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124260    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:09:11.126642 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124263    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124266    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124268    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124271    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124274    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124276    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124279    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124281    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124284    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124287    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124289    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124292    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124295    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124297    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124300    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124303    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124305    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124308    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124310    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124313    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:09:11.127116 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124315    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124317    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124320    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124323    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124325    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124328    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124331    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124333    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124336    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124339    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124342    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124344    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124347    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124349    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.124354    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 15:09:11.127589 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124479    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124485    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124488    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124491    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124494    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124497    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124500    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124502    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124505    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124508    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124511    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124514    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124517    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124520    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124522    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124525    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124528    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124530    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124533    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124535    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:09:11.127956 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124538    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124540    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124543    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124545    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124548    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124551    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124554    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124557    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124559    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124562    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124565    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124567    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124570    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124573    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124575    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124578    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124581    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124584    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124586    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124589    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:09:11.128455 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124591    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124595    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124599    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124602    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124604    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124608    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124611    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124613    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124616    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124618    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124621    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124624    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124626    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124629    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124631    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124634    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124636    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124640    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124642    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:09:11.128945 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124645    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124647    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124650    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124653    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124655    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124658    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124661    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124664    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124666    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124669    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124671    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124674    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124677    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124679    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124689    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124692    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124695    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124697    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124700    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:09:11.129424 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124704    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124708    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124712    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124715    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124718    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124721    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124723    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124726    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124729    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124732    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124735    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124738    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124740    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124743    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124745    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124748    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124751    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124753    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124756    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:09:11.129876 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124758    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:09:11.130392 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:11.124761    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:09:11.130392 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.124766    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 15:09:11.130392 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.124867    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 08 15:09:11.130392 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.126829    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 08 15:09:11.130392 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.127610    2575 server.go:1019] "Starting client certificate rotation"
Sep 08 15:09:11.130392 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.127708    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 15:09:11.130392 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.127745    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 15:09:11.130392 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.129940    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 15:09:11.131920 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.131899    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 15:09:11.140291 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.140275    2575 log.go:25] "Validated CRI v1 runtime API"
Sep 08 15:09:11.142803 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.142790    2575 log.go:25] "Validated CRI v1 image API"
Sep 08 15:09:11.143678 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.143662    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 08 15:09:11.147403 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.147371    2575 fs.go:135] Filesystem UUIDs: map[13c7cee7-44b4-4683-a1f7-5046607be0e4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d872b866-17e3-4cc2-87fd-e522f965afa2:/dev/nvme0n1p4]
Sep 08 15:09:11.147403 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.147392    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 08 15:09:11.153922 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.153800    2575 manager.go:216] Machine: {Timestamp:2026-09-08 15:09:11.152424395 +0000 UTC m=+0.312830318 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105877 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28fb3bc8e1efe210e5cde7d8f0df46 SystemUUID:ec28fb3b-c8e1-efe2-10e5-cde7d8f0df46 BootID:3c9af281-352f-4f62-9875-0ec5d7d76ce7 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:69:15:c3:ec:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:69:15:c3:ec:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8e:25:dc:dd:a8:50 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 08 15:09:11.153922 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.153909    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 08 15:09:11.154104 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.154000    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 08 15:09:11.156488 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.156464    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 08 15:09:11.156628 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.156490    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-76.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 08 15:09:11.157530 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.157510    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 15:09:11.158096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.158084    2575 topology_manager.go:138] "Creating topology manager with none policy"
Sep 08 15:09:11.158132 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.158099    2575 container_manager_linux.go:309] "Creating device plugin manager"
Sep 08 15:09:11.158132 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.158107    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 15:09:11.158781 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.158771    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 15:09:11.160018 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.160007    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 08 15:09:11.162654 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.162644    2575 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 15:09:11.162770 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.162760    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 08 15:09:11.162803 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.162789    2575 kubelet.go:486] "Attempting to sync node with API server"
Sep 08 15:09:11.162840 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.162810    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 08 15:09:11.162840 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.162830    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 08 15:09:11.162916 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.162847    2575 kubelet.go:398] "Adding apiserver pod source"
Sep 08 15:09:11.162916 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.162858    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 08 15:09:11.163359 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.163343    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 15:09:11.165002 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.164193    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 15:09:11.167487 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.167468    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 08 15:09:11.169506 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.169490    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 08 15:09:11.169566 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.169519    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 08 15:09:11.169731 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.169720    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 08 15:09:11.169770 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.169736    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 08 15:09:11.169770 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.169742    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 08 15:09:11.169770 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.169748    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 08 15:09:11.169770 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.169756    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 08 15:09:11.169770 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.169761    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 08 15:09:11.170218 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.170209    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 08 15:09:11.170218 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.170217    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 08 15:09:11.170287 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.170224    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 08 15:09:11.170287 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.170232    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 08 15:09:11.170287 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.170244    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 08 15:09:11.170287 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.170252    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 08 15:09:11.171524 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.171509    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 08 15:09:11.171524 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.171520    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 08 15:09:11.173500 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.173473    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-76.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 08 15:09:11.173591 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.173474    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 08 15:09:11.175257 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.175242    2575 server.go:1268] "Started kubelet"
Sep 08 15:09:11.175335 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.175310    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 08 15:09:11.175379 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.175319    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 08 15:09:11.175416 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.175392    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 08 15:09:11.175755 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.175738    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-76.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 08 15:09:11.176024 ip-10-0-132-76 systemd[1]: Started Kubernetes Kubelet.
Sep 08 15:09:11.176549 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.176532    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 08 15:09:11.178626 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.177950    2575 server.go:310] "Adding debug handlers to kubelet server"
Sep 08 15:09:11.181797 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.181776    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 08 15:09:11.182731 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.182713    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 08 15:09:11.182731 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.182726    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 08 15:09:11.182915 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.182896    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-76.ec2.internal\" not found"
Sep 08 15:09:11.183235 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.183220    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 08 15:09:11.183327 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.183280    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 08 15:09:11.183327 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.183293    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 08 15:09:11.183408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.183365    2575 reconstruct.go:97] "Volume reconstruction finished"
Sep 08 15:09:11.183408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.183375    2575 reconciler.go:29] "Reconciler: start to sync state"
Sep 08 15:09:11.183706 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.183689    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 08 15:09:11.183771 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.183709    2575 factory.go:55] Registering systemd factory
Sep 08 15:09:11.183771 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.183720    2575 factory.go:223] Registration of the systemd container factory successfully
Sep 08 15:09:11.183984 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.183967    2575 factory.go:153] Registering CRI-O factory
Sep 08 15:09:11.183984 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.183985    2575 factory.go:223] Registration of the crio container factory successfully
Sep 08 15:09:11.184139 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.184079    2575 factory.go:103] Registering Raw factory
Sep 08 15:09:11.184139 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.184093    2575 manager.go:1196] Started watching for new ooms in manager
Sep 08 15:09:11.184460 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.184445    2575 manager.go:349] Starting recovery of all containers
Sep 08 15:09:11.185426 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.185386    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-76.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 08 15:09:11.185529 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.184595    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-76.ec2.internal.18d36105048bc5a9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-76.ec2.internal,UID:ip-10-0-132-76.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-76.ec2.internal,},FirstTimestamp:2026-09-08 15:09:11.175218601 +0000 UTC m=+0.335624524,LastTimestamp:2026-09-08 15:09:11.175218601 +0000 UTC m=+0.335624524,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-76.ec2.internal,}"
Sep 08 15:09:11.186789 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.186750    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 08 15:09:11.190890 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.190859    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ktb5q"
Sep 08 15:09:11.195467 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.194176    2575 manager.go:354] Recovery completed
Sep 08 15:09:11.199533 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.199390    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ktb5q"
Sep 08 15:09:11.200822 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.200778    2575 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 08 15:09:11.204088 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.204074    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:09:11.205666 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.205651    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:09:11.205735 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.205679    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:09:11.205735 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.205693    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:09:11.206231 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.206216    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 08 15:09:11.206231 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.206231    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 08 15:09:11.206338 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.206233    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:09:11.206338 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.206246    2575 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 15:09:11.206970 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.206947    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:09:11.207049 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.206985    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:09:11.207049 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.207000    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:09:11.283674 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.283656    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-76.ec2.internal\" not found"
Sep 08 15:09:11.347152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.347139    2575 policy_none.go:49] "None policy: Start"
Sep 08 15:09:11.347215 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.347156    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 08 15:09:11.347215 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.347167    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 08 15:09:11.348123 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.348107    2575 policy_none.go:47] "Start"
Sep 08 15:09:11.349194 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.349172    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 08 15:09:11.350587 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.350562    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 08 15:09:11.350671 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.350624    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 08 15:09:11.350671 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.350647    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 08 15:09:11.350775 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.350718    2575 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 08 15:09:11.353262 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.353240    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:09:11.374539 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.374525    2575 manager.go:337] "Starting Device Plugin manager"
Sep 08 15:09:11.374605 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.374556    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 08 15:09:11.374605 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.374565    2575 server.go:87] "Starting device plugin registration server"
Sep 08 15:09:11.374850 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.374838    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 08 15:09:11.374919 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.374851    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 08 15:09:11.374977 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.374936    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 08 15:09:11.375026 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.375001    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 08 15:09:11.375026 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.375006    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 08 15:09:11.375501 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.375479    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 08 15:09:11.375562 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.375539    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-76.ec2.internal\" not found"
Sep 08 15:09:11.393437 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.393420    2575 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-76.ec2.internal\" not found" node="ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.451571 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.451547    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-76.ec2.internal"]
Sep 08 15:09:11.451641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.451631    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:09:11.453087 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.453068    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:09:11.453194 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.453099    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:09:11.453194 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.453112    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:09:11.454195 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.454184    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:09:11.454313 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.454298    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.454414 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.454341    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:09:11.454868 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.454853    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:09:11.454868 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.454865    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:09:11.454984 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.454882    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:09:11.454984 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.454883    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:09:11.454984 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.454906    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:09:11.454984 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.454892    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:09:11.456143 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.456128    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.456222 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.456157    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:09:11.456813 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.456799    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:09:11.456877 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.456822    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:09:11.456877 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.456835    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:09:11.475724 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.475697    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:09:11.476348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.476334    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:09:11.476418 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.476357    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:09:11.476418 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.476370    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:09:11.476418 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.476386    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.477577 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.477564    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-76.ec2.internal\" not found" node="ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.481421 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.481407    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-76.ec2.internal\" not found" node="ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.482780 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.482768    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.482823 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.482785    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-76.ec2.internal\": node \"ip-10-0-132-76.ec2.internal\" not found"
Sep 08 15:09:11.484215 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.484201    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/87be9da9c6492bc0bdf474fd1bdc742a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal\" (UID: \"87be9da9c6492bc0bdf474fd1bdc742a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.484270 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.484223    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87be9da9c6492bc0bdf474fd1bdc742a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal\" (UID: \"87be9da9c6492bc0bdf474fd1bdc742a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.484270 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.484236    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b674e83ab0483a337814cf332a4de8c1-config\") pod \"kube-apiserver-proxy-ip-10-0-132-76.ec2.internal\" (UID: \"b674e83ab0483a337814cf332a4de8c1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.500909 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.500888    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-76.ec2.internal\" not found"
Sep 08 15:09:11.584341 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.584318    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/87be9da9c6492bc0bdf474fd1bdc742a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal\" (UID: \"87be9da9c6492bc0bdf474fd1bdc742a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.584415 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.584341    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87be9da9c6492bc0bdf474fd1bdc742a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal\" (UID: \"87be9da9c6492bc0bdf474fd1bdc742a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.584415 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.584361    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b674e83ab0483a337814cf332a4de8c1-config\") pod \"kube-apiserver-proxy-ip-10-0-132-76.ec2.internal\" (UID: \"b674e83ab0483a337814cf332a4de8c1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.584415 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.584405    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/87be9da9c6492bc0bdf474fd1bdc742a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal\" (UID: \"87be9da9c6492bc0bdf474fd1bdc742a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.584509 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.584420    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87be9da9c6492bc0bdf474fd1bdc742a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal\" (UID: \"87be9da9c6492bc0bdf474fd1bdc742a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.584509 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.584444    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b674e83ab0483a337814cf332a4de8c1-config\") pod \"kube-apiserver-proxy-ip-10-0-132-76.ec2.internal\" (UID: \"b674e83ab0483a337814cf332a4de8c1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.601469 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.601447    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-76.ec2.internal\" not found"
Sep 08 15:09:11.701697 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.701644    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-76.ec2.internal\" not found"
Sep 08 15:09:11.779852 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.779822    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.783679 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:11.783618    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:11.802261 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.802241    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-76.ec2.internal\" not found"
Sep 08 15:09:11.902742 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:11.902716    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-76.ec2.internal\" not found"
Sep 08 15:09:12.003225 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:12.003170    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-76.ec2.internal\" not found"
Sep 08 15:09:13.178131 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:13.178106    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-76.ec2.internal\" not found"
Sep 08 15:09:13.202264 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.202246    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 08 15:09:13.202651 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.202373    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 15:09:13.241823 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.241795    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:09:13.257932 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.257911    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:13.257998 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.257949    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 08 15:09:13.268429 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.268405    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 15:09:13.270074 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.270059    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal"
Sep 08 15:09:13.273548 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.273530    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 15:09:13.276551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.276521    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 15:04:12 +0000 UTC" deadline="2028-03-04 02:15:34.238967508 +0000 UTC"
Sep 08 15:09:13.276603 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.276552    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13019h6m20.962418667s"
Sep 08 15:09:13.282452 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.282438    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 15:09:13.296412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.296394    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bfkj8"
Sep 08 15:09:13.305086 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.305066    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bfkj8"
Sep 08 15:09:13.325789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.325770    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:09:13.657504 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:13.657448    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87be9da9c6492bc0bdf474fd1bdc742a.slice/crio-b37b2523e326c27d9cf169641dd259e9e3052dc4454683be8c59c6a395f20e0c WatchSource:0}: Error finding container b37b2523e326c27d9cf169641dd259e9e3052dc4454683be8c59c6a395f20e0c: Status 404 returned error can't find the container with id b37b2523e326c27d9cf169641dd259e9e3052dc4454683be8c59c6a395f20e0c
Sep 08 15:09:13.657711 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:13.657694    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb674e83ab0483a337814cf332a4de8c1.slice/crio-ad6ac860a5da7e3c746bb3488b6113aea5b0995e674afd6cdc894fc8d5624cd6 WatchSource:0}: Error finding container ad6ac860a5da7e3c746bb3488b6113aea5b0995e674afd6cdc894fc8d5624cd6: Status 404 returned error can't find the container with id ad6ac860a5da7e3c746bb3488b6113aea5b0995e674afd6cdc894fc8d5624cd6
Sep 08 15:09:13.661730 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.661715    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:09:13.715336 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:13.715316    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:09:14.172270 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.172244    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:09:14.238777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.238757    2575 apiserver.go:52] "Watching apiserver"
Sep 08 15:09:14.247757 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.247733    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 08 15:09:14.248157 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.248132    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6","openshift-cluster-node-tuning-operator/tuned-727ns","openshift-dns/node-resolver-4ltq9","openshift-image-registry/node-ca-qvxxd","openshift-multus/network-metrics-daemon-9vwnd","openshift-network-operator/iptables-alerter-vx9n2","kube-system/konnectivity-agent-8fh7x","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal","openshift-multus/multus-additional-cni-plugins-fchvk","openshift-multus/multus-zrlxq","openshift-network-diagnostics/network-check-target-dlpvm","openshift-ovn-kubernetes/ovnkube-node-9w9cc","kube-system/global-pull-secret-syncer-lbq79","kube-system/kube-apiserver-proxy-ip-10-0-132-76.ec2.internal"]
Sep 08 15:09:14.250265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.250247    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8fh7x"
Sep 08 15:09:14.251281 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.251254    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:14.251378 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.251336    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:14.252284 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.252264    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4ltq9"
Sep 08 15:09:14.253149 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.253130    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 08 15:09:14.253149 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.253128    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 08 15:09:14.253298 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.253135    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gs57x\""
Sep 08 15:09:14.253352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.253294    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qvxxd"
Sep 08 15:09:14.254586 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.254555    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 08 15:09:14.254702 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.254649    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:14.254702 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.254669    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 08 15:09:14.254910 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.254718    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:14.254910 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.254790    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cbxvm\""
Sep 08 15:09:14.255723 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.255702    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 08 15:09:14.255780 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.255702    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 08 15:09:14.255980 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.255965    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 08 15:09:14.256602 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.256582    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vx9n2"
Sep 08 15:09:14.257085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.257068    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wkfsh\""
Sep 08 15:09:14.258934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.258889    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.259034 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.258993    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 08 15:09:14.259231 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.258999    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 08 15:09:14.259540 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.259524    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:09:14.259648 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.259593    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8xm5w\""
Sep 08 15:09:14.260162 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.260146    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.261305 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.261286    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.261440 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.261420    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 08 15:09:14.261820 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.261797    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 08 15:09:14.261923 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.261807    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 08 15:09:14.261923 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.261869    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 08 15:09:14.262188 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.262171    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6c75w\""
Sep 08 15:09:14.262253 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.262234    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 08 15:09:14.262612 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.262594    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.263091 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.262726    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 08 15:09:14.263091 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.262726    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-thd27\""
Sep 08 15:09:14.263091 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.262729    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 08 15:09:14.263091 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.262802    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 08 15:09:14.263859 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.263840    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:14.263956 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.263893    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:14.265052 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265034    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.265521 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265479    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb368e23-9846-425c-be55-cee17f483954-host-slash\") pod \"iptables-alerter-vx9n2\" (UID: \"cb368e23-9846-425c-be55-cee17f483954\") " pod="openshift-network-operator/iptables-alerter-vx9n2"
Sep 08 15:09:14.265621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265549    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d33f854f-48e2-45ca-8289-0f5cb0817be7-hosts-file\") pod \"node-resolver-4ltq9\" (UID: \"d33f854f-48e2-45ca-8289-0f5cb0817be7\") " pod="openshift-dns/node-resolver-4ltq9"
Sep 08 15:09:14.265621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265577    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrsfj\" (UniqueName: \"kubernetes.io/projected/d33f854f-48e2-45ca-8289-0f5cb0817be7-kube-api-access-jrsfj\") pod \"node-resolver-4ltq9\" (UID: \"d33f854f-48e2-45ca-8289-0f5cb0817be7\") " pod="openshift-dns/node-resolver-4ltq9"
Sep 08 15:09:14.265621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265604    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9b25500-9302-461f-9458-f2e4a5dfbcd7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.265720 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265629    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d64afe66-e605-40e6-a64f-11dd8f2165cf-konnectivity-ca\") pod \"konnectivity-agent-8fh7x\" (UID: \"d64afe66-e605-40e6-a64f-11dd8f2165cf\") " pod="kube-system/konnectivity-agent-8fh7x"
Sep 08 15:09:14.265720 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265656    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:14.265720 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265680    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dea9369e-6c20-4b59-a04b-eb20d2f8cca5-serviceca\") pod \"node-ca-qvxxd\" (UID: \"dea9369e-6c20-4b59-a04b-eb20d2f8cca5\") " pod="openshift-image-registry/node-ca-qvxxd"
Sep 08 15:09:14.265720 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265702    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8mj2\" (UniqueName: \"kubernetes.io/projected/dea9369e-6c20-4b59-a04b-eb20d2f8cca5-kube-api-access-n8mj2\") pod \"node-ca-qvxxd\" (UID: \"dea9369e-6c20-4b59-a04b-eb20d2f8cca5\") " pod="openshift-image-registry/node-ca-qvxxd"
Sep 08 15:09:14.265839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265727    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9b25500-9302-461f-9458-f2e4a5dfbcd7-system-cni-dir\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.265839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265778    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9b25500-9302-461f-9458-f2e4a5dfbcd7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.265839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265816    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjwvz\" (UniqueName: \"kubernetes.io/projected/e9b25500-9302-461f-9458-f2e4a5dfbcd7-kube-api-access-fjwvz\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.265966 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265846    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:14.265966 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265875    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzwn9\" (UniqueName: \"kubernetes.io/projected/4f348729-57f1-40ab-ae97-85bfefa7d80e-kube-api-access-qzwn9\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:14.265966 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265902    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d33f854f-48e2-45ca-8289-0f5cb0817be7-tmp-dir\") pod \"node-resolver-4ltq9\" (UID: \"d33f854f-48e2-45ca-8289-0f5cb0817be7\") " pod="openshift-dns/node-resolver-4ltq9"
Sep 08 15:09:14.265966 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265930    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dea9369e-6c20-4b59-a04b-eb20d2f8cca5-host\") pod \"node-ca-qvxxd\" (UID: \"dea9369e-6c20-4b59-a04b-eb20d2f8cca5\") " pod="openshift-image-registry/node-ca-qvxxd"
Sep 08 15:09:14.265966 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265954    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9b25500-9302-461f-9458-f2e4a5dfbcd7-cni-binary-copy\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.266260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.265980    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9b25500-9302-461f-9458-f2e4a5dfbcd7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.266260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.266003    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/18237a47-8e2c-42f5-b960-f56188420981-dbus\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:14.266260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.266036    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cb368e23-9846-425c-be55-cee17f483954-iptables-alerter-script\") pod \"iptables-alerter-vx9n2\" (UID: \"cb368e23-9846-425c-be55-cee17f483954\") " pod="openshift-network-operator/iptables-alerter-vx9n2"
Sep 08 15:09:14.266260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.266083    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r8qr\" (UniqueName: \"kubernetes.io/projected/cb368e23-9846-425c-be55-cee17f483954-kube-api-access-7r8qr\") pod \"iptables-alerter-vx9n2\" (UID: \"cb368e23-9846-425c-be55-cee17f483954\") " pod="openshift-network-operator/iptables-alerter-vx9n2"
Sep 08 15:09:14.266260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.266154    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9b25500-9302-461f-9458-f2e4a5dfbcd7-cnibin\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.266260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.266181    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d64afe66-e605-40e6-a64f-11dd8f2165cf-agent-certs\") pod \"konnectivity-agent-8fh7x\" (UID: \"d64afe66-e605-40e6-a64f-11dd8f2165cf\") " pod="kube-system/konnectivity-agent-8fh7x"
Sep 08 15:09:14.266260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.266206    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/18237a47-8e2c-42f5-b960-f56188420981-kubelet-config\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:14.267222 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.267202    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 08 15:09:14.267352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.267334    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:09:14.268169 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.268151    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 08 15:09:14.268238 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.268171    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 08 15:09:14.268883 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.268867    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cq9bf\""
Sep 08 15:09:14.269725 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.269651    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 08 15:09:14.269909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.269878    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 08 15:09:14.270031 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.270002    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-shrg8\""
Sep 08 15:09:14.270347 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.270088    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7trxz\""
Sep 08 15:09:14.270347 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.270150    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 08 15:09:14.270657 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.270635    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 08 15:09:14.271189 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.271164    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 08 15:09:14.307279 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.307234    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 15:04:13 +0000 UTC" deadline="2028-03-03 00:20:41.402142797 +0000 UTC"
Sep 08 15:09:14.307369 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.307279    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12993h11m27.094867876s"
Sep 08 15:09:14.358368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.358345    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 08 15:09:14.367099 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367081    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d33f854f-48e2-45ca-8289-0f5cb0817be7-tmp-dir\") pod \"node-resolver-4ltq9\" (UID: \"d33f854f-48e2-45ca-8289-0f5cb0817be7\") " pod="openshift-dns/node-resolver-4ltq9"
Sep 08 15:09:14.367181 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367105    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dea9369e-6c20-4b59-a04b-eb20d2f8cca5-host\") pod \"node-ca-qvxxd\" (UID: \"dea9369e-6c20-4b59-a04b-eb20d2f8cca5\") " pod="openshift-image-registry/node-ca-qvxxd"
Sep 08 15:09:14.367181 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367120    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9b25500-9302-461f-9458-f2e4a5dfbcd7-cni-binary-copy\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.367181 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367134    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/18237a47-8e2c-42f5-b960-f56188420981-dbus\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:14.367181 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367158    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.367181 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367173    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-systemd-units\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.367408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367202    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dea9369e-6c20-4b59-a04b-eb20d2f8cca5-host\") pod \"node-ca-qvxxd\" (UID: \"dea9369e-6c20-4b59-a04b-eb20d2f8cca5\") " pod="openshift-image-registry/node-ca-qvxxd"
Sep 08 15:09:14.367408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367229    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/18237a47-8e2c-42f5-b960-f56188420981-dbus\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:14.367408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367233    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-log-socket\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.367408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367268    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-modprobe-d\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.367408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367297    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d64afe66-e605-40e6-a64f-11dd8f2165cf-agent-certs\") pod \"konnectivity-agent-8fh7x\" (UID: \"d64afe66-e605-40e6-a64f-11dd8f2165cf\") " pod="kube-system/konnectivity-agent-8fh7x"
Sep 08 15:09:14.367408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367338    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9zms\" (UniqueName: \"kubernetes.io/projected/e13f4f32-5641-4b50-8b14-e94551c552c0-kube-api-access-j9zms\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.367675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367452    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6586e3c3-662b-42dd-9891-3b6c94042172-ovn-node-metrics-cert\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.367675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367506    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-kubernetes\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.367675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367532    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0011d520-9d6d-4932-91f9-80084021b951-tmp\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.367675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367574    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb368e23-9846-425c-be55-cee17f483954-host-slash\") pod \"iptables-alerter-vx9n2\" (UID: \"cb368e23-9846-425c-be55-cee17f483954\") " pod="openshift-network-operator/iptables-alerter-vx9n2"
Sep 08 15:09:14.367675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367601    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d64afe66-e605-40e6-a64f-11dd8f2165cf-konnectivity-ca\") pod \"konnectivity-agent-8fh7x\" (UID: \"d64afe66-e605-40e6-a64f-11dd8f2165cf\") " pod="kube-system/konnectivity-agent-8fh7x"
Sep 08 15:09:14.367675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367618    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-cnibin\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.367675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367620    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb368e23-9846-425c-be55-cee17f483954-host-slash\") pod \"iptables-alerter-vx9n2\" (UID: \"cb368e23-9846-425c-be55-cee17f483954\") " pod="openshift-network-operator/iptables-alerter-vx9n2"
Sep 08 15:09:14.367675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367648    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-run-ovn-kubernetes\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367729    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-sysctl-d\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367750    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-host\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367774    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dea9369e-6c20-4b59-a04b-eb20d2f8cca5-serviceca\") pod \"node-ca-qvxxd\" (UID: \"dea9369e-6c20-4b59-a04b-eb20d2f8cca5\") " pod="openshift-image-registry/node-ca-qvxxd"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367795    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-run-ovn\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367818    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367841    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h9bt\" (UniqueName: \"kubernetes.io/projected/6586e3c3-662b-42dd-9891-3b6c94042172-kube-api-access-2h9bt\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367865    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-run-netns\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367878    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d33f854f-48e2-45ca-8289-0f5cb0817be7-tmp-dir\") pod \"node-resolver-4ltq9\" (UID: \"d33f854f-48e2-45ca-8289-0f5cb0817be7\") " pod="openshift-dns/node-resolver-4ltq9"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367906    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-run-systemd\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367930    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6586e3c3-662b-42dd-9891-3b6c94042172-ovnkube-script-lib\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367953    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-sys\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367977    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0011d520-9d6d-4932-91f9-80084021b951-etc-tuned\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367993    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d64afe66-e605-40e6-a64f-11dd8f2165cf-konnectivity-ca\") pod \"konnectivity-agent-8fh7x\" (UID: \"d64afe66-e605-40e6-a64f-11dd8f2165cf\") " pod="kube-system/konnectivity-agent-8fh7x"
Sep 08 15:09:14.367997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.367994    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368034    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r8qr\" (UniqueName: \"kubernetes.io/projected/cb368e23-9846-425c-be55-cee17f483954-kube-api-access-7r8qr\") pod \"iptables-alerter-vx9n2\" (UID: \"cb368e23-9846-425c-be55-cee17f483954\") " pod="openshift-network-operator/iptables-alerter-vx9n2"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368066    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-kubelet\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368151    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dea9369e-6c20-4b59-a04b-eb20d2f8cca5-serviceca\") pod \"node-ca-qvxxd\" (UID: \"dea9369e-6c20-4b59-a04b-eb20d2f8cca5\") " pod="openshift-image-registry/node-ca-qvxxd"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368171    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-sysctl-conf\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368199    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-registration-dir\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368189    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9b25500-9302-461f-9458-f2e4a5dfbcd7-cni-binary-copy\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368224    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-hostroot\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368249    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4rxx\" (UniqueName: \"kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx\") pod \"network-check-target-dlpvm\" (UID: \"7ccbb705-396e-4f0f-8756-b1a81ae1a840\") " pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368276    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-slash\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368308    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-node-log\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368334    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368351    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8mj2\" (UniqueName: \"kubernetes.io/projected/dea9369e-6c20-4b59-a04b-eb20d2f8cca5-kube-api-access-n8mj2\") pod \"node-ca-qvxxd\" (UID: \"dea9369e-6c20-4b59-a04b-eb20d2f8cca5\") " pod="openshift-image-registry/node-ca-qvxxd"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368366    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e13f4f32-5641-4b50-8b14-e94551c552c0-multus-daemon-config\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368381    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-cni-bin\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368402    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-var-lib-kubelet\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368424    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-system-cni-dir\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.368551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368457    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-multus-cni-dir\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.368466    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368501    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e13f4f32-5641-4b50-8b14-e94551c552c0-cni-binary-copy\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368530    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-multus-conf-dir\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368561    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-var-lib-openvswitch\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.368580    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs podName:4f348729-57f1-40ab-ae97-85bfefa7d80e nodeName:}" failed. No retries permitted until 2026-09-08 15:09:14.868559566 +0000 UTC m=+2.954509609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs") pod "network-metrics-daemon-9vwnd" (UID: "4f348729-57f1-40ab-ae97-85bfefa7d80e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368641    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-sysconfig\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368672    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcs5b\" (UniqueName: \"kubernetes.io/projected/0011d520-9d6d-4932-91f9-80084021b951-kube-api-access-vcs5b\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368721    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d33f854f-48e2-45ca-8289-0f5cb0817be7-hosts-file\") pod \"node-resolver-4ltq9\" (UID: \"d33f854f-48e2-45ca-8289-0f5cb0817be7\") " pod="openshift-dns/node-resolver-4ltq9"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368765    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d33f854f-48e2-45ca-8289-0f5cb0817be7-hosts-file\") pod \"node-resolver-4ltq9\" (UID: \"d33f854f-48e2-45ca-8289-0f5cb0817be7\") " pod="openshift-dns/node-resolver-4ltq9"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368773    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-etc-selinux\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368800    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-var-lib-cni-multus\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368822    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-run-netns\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368840    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6586e3c3-662b-42dd-9891-3b6c94042172-env-overrides\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368864    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9b25500-9302-461f-9458-f2e4a5dfbcd7-system-cni-dir\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368891    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:14.369248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368914    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-socket-dir\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368932    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9b25500-9302-461f-9458-f2e4a5dfbcd7-system-cni-dir\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368932    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-device-dir\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368959    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-run-k8s-cni-cncf-io\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.368977    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.368995    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-var-lib-cni-bin\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.369018    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret podName:18237a47-8e2c-42f5-b960-f56188420981 nodeName:}" failed. No retries permitted until 2026-09-08 15:09:14.869006849 +0000 UTC m=+2.954956890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret") pod "global-pull-secret-syncer-lbq79" (UID: "18237a47-8e2c-42f5-b960-f56188420981") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369042    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-etc-kubernetes\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369085    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-etc-openvswitch\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369124    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzwn9\" (UniqueName: \"kubernetes.io/projected/4f348729-57f1-40ab-ae97-85bfefa7d80e-kube-api-access-qzwn9\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369164    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9b25500-9302-461f-9458-f2e4a5dfbcd7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369259    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-run-multus-certs\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369289    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-systemd\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369315    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cb368e23-9846-425c-be55-cee17f483954-iptables-alerter-script\") pod \"iptables-alerter-vx9n2\" (UID: \"cb368e23-9846-425c-be55-cee17f483954\") " pod="openshift-network-operator/iptables-alerter-vx9n2"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369353    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9b25500-9302-461f-9458-f2e4a5dfbcd7-cnibin\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369393    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/18237a47-8e2c-42f5-b960-f56188420981-kubelet-config\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:14.369975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369411    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9b25500-9302-461f-9458-f2e4a5dfbcd7-cnibin\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369438    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-multus-socket-dir-parent\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369458    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-var-lib-kubelet\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369516    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/18237a47-8e2c-42f5-b960-f56188420981-kubelet-config\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369514    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-run-openvswitch\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369543    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-cni-netd\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369557    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-run\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369569    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9b25500-9302-461f-9458-f2e4a5dfbcd7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369588    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrsfj\" (UniqueName: \"kubernetes.io/projected/d33f854f-48e2-45ca-8289-0f5cb0817be7-kube-api-access-jrsfj\") pod \"node-resolver-4ltq9\" (UID: \"d33f854f-48e2-45ca-8289-0f5cb0817be7\") " pod="openshift-dns/node-resolver-4ltq9"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369617    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9b25500-9302-461f-9458-f2e4a5dfbcd7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369642    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8rr9\" (UniqueName: \"kubernetes.io/projected/75b986d3-6372-4117-94d8-7f26bcc542da-kube-api-access-r8rr9\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369729    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-lib-modules\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369756    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9b25500-9302-461f-9458-f2e4a5dfbcd7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369782    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjwvz\" (UniqueName: \"kubernetes.io/projected/e9b25500-9302-461f-9458-f2e4a5dfbcd7-kube-api-access-fjwvz\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369803    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-sys-fs\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369818    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6586e3c3-662b-42dd-9891-3b6c94042172-ovnkube-config\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.370423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369824    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cb368e23-9846-425c-be55-cee17f483954-iptables-alerter-script\") pod \"iptables-alerter-vx9n2\" (UID: \"cb368e23-9846-425c-be55-cee17f483954\") " pod="openshift-network-operator/iptables-alerter-vx9n2"
Sep 08 15:09:14.370890 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.369933    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9b25500-9302-461f-9458-f2e4a5dfbcd7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.370890 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.370044    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9b25500-9302-461f-9458-f2e4a5dfbcd7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.371427 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.371411    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d64afe66-e605-40e6-a64f-11dd8f2165cf-agent-certs\") pod \"konnectivity-agent-8fh7x\" (UID: \"d64afe66-e605-40e6-a64f-11dd8f2165cf\") " pod="kube-system/konnectivity-agent-8fh7x"
Sep 08 15:09:14.386000 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.385982    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r8qr\" (UniqueName: \"kubernetes.io/projected/cb368e23-9846-425c-be55-cee17f483954-kube-api-access-7r8qr\") pod \"iptables-alerter-vx9n2\" (UID: \"cb368e23-9846-425c-be55-cee17f483954\") " pod="openshift-network-operator/iptables-alerter-vx9n2"
Sep 08 15:09:14.387392 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.387368    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrsfj\" (UniqueName: \"kubernetes.io/projected/d33f854f-48e2-45ca-8289-0f5cb0817be7-kube-api-access-jrsfj\") pod \"node-resolver-4ltq9\" (UID: \"d33f854f-48e2-45ca-8289-0f5cb0817be7\") " pod="openshift-dns/node-resolver-4ltq9"
Sep 08 15:09:14.388381 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.388365    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzwn9\" (UniqueName: \"kubernetes.io/projected/4f348729-57f1-40ab-ae97-85bfefa7d80e-kube-api-access-qzwn9\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:14.388614 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.388597    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjwvz\" (UniqueName: \"kubernetes.io/projected/e9b25500-9302-461f-9458-f2e4a5dfbcd7-kube-api-access-fjwvz\") pod \"multus-additional-cni-plugins-fchvk\" (UID: \"e9b25500-9302-461f-9458-f2e4a5dfbcd7\") " pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.388861 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.388845    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8mj2\" (UniqueName: \"kubernetes.io/projected/dea9369e-6c20-4b59-a04b-eb20d2f8cca5-kube-api-access-n8mj2\") pod \"node-ca-qvxxd\" (UID: \"dea9369e-6c20-4b59-a04b-eb20d2f8cca5\") " pod="openshift-image-registry/node-ca-qvxxd"
Sep 08 15:09:14.431067 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.431004    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal" event={"ID":"87be9da9c6492bc0bdf474fd1bdc742a","Type":"ContainerStarted","Data":"b37b2523e326c27d9cf169641dd259e9e3052dc4454683be8c59c6a395f20e0c"}
Sep 08 15:09:14.431953 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.431937    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-76.ec2.internal" event={"ID":"b674e83ab0483a337814cf332a4de8c1","Type":"ContainerStarted","Data":"ad6ac860a5da7e3c746bb3488b6113aea5b0995e674afd6cdc894fc8d5624cd6"}
Sep 08 15:09:14.470762 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470738    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-run-netns\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.470850 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470765    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-run-systemd\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.470850 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470780    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6586e3c3-662b-42dd-9891-3b6c94042172-ovnkube-script-lib\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.470850 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470810    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-sys\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.470850 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470828    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-run-netns\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.470850 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470834    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0011d520-9d6d-4932-91f9-80084021b951-etc-tuned\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470872    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-kubelet\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470884    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-sys\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470898    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-sysctl-conf\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470835    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-run-systemd\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470922    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-registration-dir\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470947    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-hostroot\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470962    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-kubelet\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.470970    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rxx\" (UniqueName: \"kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx\") pod \"network-check-target-dlpvm\" (UID: \"7ccbb705-396e-4f0f-8756-b1a81ae1a840\") " pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471015    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-slash\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471018    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-sysctl-conf\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471032    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-hostroot\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471043    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-node-log\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471009    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-registration-dir\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.471081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471065    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-slash\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471168    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-node-log\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471340    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6586e3c3-662b-42dd-9891-3b6c94042172-ovnkube-script-lib\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471363    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e13f4f32-5641-4b50-8b14-e94551c552c0-multus-daemon-config\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471395    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-cni-bin\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471417    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-var-lib-kubelet\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471439    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-system-cni-dir\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471459    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-cni-bin\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471462    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-multus-cni-dir\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471510    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e13f4f32-5641-4b50-8b14-e94551c552c0-cni-binary-copy\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471512    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-var-lib-kubelet\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471533    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-multus-cni-dir\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471535    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-multus-conf-dir\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471551    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-system-cni-dir\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471564    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-var-lib-openvswitch\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471566    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-multus-conf-dir\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471604    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-sysconfig\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471620    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-var-lib-openvswitch\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.471675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471627    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcs5b\" (UniqueName: \"kubernetes.io/projected/0011d520-9d6d-4932-91f9-80084021b951-kube-api-access-vcs5b\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471669    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-sysconfig\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471697    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-etc-selinux\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471733    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-var-lib-cni-multus\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471759    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-run-netns\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471783    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6586e3c3-662b-42dd-9891-3b6c94042172-env-overrides\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471788    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-etc-selinux\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471790    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-var-lib-cni-multus\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471820    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-socket-dir\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471844    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-device-dir\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471850    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-run-netns\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471867    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-run-k8s-cni-cncf-io\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471891    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-var-lib-cni-bin\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471914    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-etc-kubernetes\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471933    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-device-dir\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471936    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-etc-openvswitch\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471977    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-etc-openvswitch\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.472412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.471979    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-socket-dir\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472002    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-run-multus-certs\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472015    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-var-lib-cni-bin\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472028    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-systemd\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472032    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-etc-kubernetes\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472038    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e13f4f32-5641-4b50-8b14-e94551c552c0-cni-binary-copy\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472060    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-run-k8s-cni-cncf-io\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472070    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-run-multus-certs\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e13f4f32-5641-4b50-8b14-e94551c552c0-multus-daemon-config\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472079    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-multus-socket-dir-parent\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472106    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-systemd\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472110    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-multus-socket-dir-parent\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472127    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-var-lib-kubelet\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472142    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6586e3c3-662b-42dd-9891-3b6c94042172-env-overrides\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-run-openvswitch\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472181    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-cni-netd\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472187    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-run-openvswitch\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472205    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-run\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.473136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472225    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-host-var-lib-kubelet\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472231    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8rr9\" (UniqueName: \"kubernetes.io/projected/75b986d3-6372-4117-94d8-7f26bcc542da-kube-api-access-r8rr9\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472259    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-lib-modules\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472264    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-run\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472229    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-cni-netd\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472386    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-lib-modules\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472428    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-sys-fs\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472456    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6586e3c3-662b-42dd-9891-3b6c94042172-ovnkube-config\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472472    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472487    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-systemd-units\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472525    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-sys-fs\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472518    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-log-socket\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472558    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/75b986d3-6372-4117-94d8-7f26bcc542da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472572    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-log-socket\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472563    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-systemd-units\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472574    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-modprobe-d\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472616    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9zms\" (UniqueName: \"kubernetes.io/projected/e13f4f32-5641-4b50-8b14-e94551c552c0-kube-api-access-j9zms\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.473708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472642    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6586e3c3-662b-42dd-9891-3b6c94042172-ovn-node-metrics-cert\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472665    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-kubernetes\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472688    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0011d520-9d6d-4932-91f9-80084021b951-tmp\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472698    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-modprobe-d\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472714    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-cnibin\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472737    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-run-ovn-kubernetes\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472751    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-kubernetes\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472759    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-sysctl-d\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472799    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-host\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472803    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e13f4f32-5641-4b50-8b14-e94551c552c0-cnibin\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472822    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-run-ovn\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472847    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472872    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h9bt\" (UniqueName: \"kubernetes.io/projected/6586e3c3-662b-42dd-9891-3b6c94042172-kube-api-access-2h9bt\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472891    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-etc-sysctl-d\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472939    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-run-ovn\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472948    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6586e3c3-662b-42dd-9891-3b6c94042172-ovnkube-config\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472939    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6586e3c3-662b-42dd-9891-3b6c94042172-host-run-ovn-kubernetes\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.474223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.472988    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0011d520-9d6d-4932-91f9-80084021b951-host\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.474884 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.473006    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0011d520-9d6d-4932-91f9-80084021b951-etc-tuned\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.474884 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.474618    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0011d520-9d6d-4932-91f9-80084021b951-tmp\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.474884 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.474844    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6586e3c3-662b-42dd-9891-3b6c94042172-ovn-node-metrics-cert\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.477770 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.477753    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:09:14.477854 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.477778    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:09:14.477854 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.477791    2575 projected.go:196] Error preparing data for projected volume kube-api-access-g4rxx for pod openshift-network-diagnostics/network-check-target-dlpvm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:09:14.477854 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.477850    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx podName:7ccbb705-396e-4f0f-8756-b1a81ae1a840 nodeName:}" failed. No retries permitted until 2026-09-08 15:09:14.977835464 +0000 UTC m=+3.063785486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-g4rxx" (UniqueName: "kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx") pod "network-check-target-dlpvm" (UID: "7ccbb705-396e-4f0f-8756-b1a81ae1a840") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:09:14.480043 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.480017    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcs5b\" (UniqueName: \"kubernetes.io/projected/0011d520-9d6d-4932-91f9-80084021b951-kube-api-access-vcs5b\") pod \"tuned-727ns\" (UID: \"0011d520-9d6d-4932-91f9-80084021b951\") " pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.480973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.480782    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8rr9\" (UniqueName: \"kubernetes.io/projected/75b986d3-6372-4117-94d8-7f26bcc542da-kube-api-access-r8rr9\") pod \"aws-ebs-csi-driver-node-bwkj6\" (UID: \"75b986d3-6372-4117-94d8-7f26bcc542da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.481271 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.481250    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h9bt\" (UniqueName: \"kubernetes.io/projected/6586e3c3-662b-42dd-9891-3b6c94042172-kube-api-access-2h9bt\") pod \"ovnkube-node-9w9cc\" (UID: \"6586e3c3-662b-42dd-9891-3b6c94042172\") " pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.481341 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.481273    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9zms\" (UniqueName: \"kubernetes.io/projected/e13f4f32-5641-4b50-8b14-e94551c552c0-kube-api-access-j9zms\") pod \"multus-zrlxq\" (UID: \"e13f4f32-5641-4b50-8b14-e94551c552c0\") " pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.561442 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.561420    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8fh7x"
Sep 08 15:09:14.569211 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.569145    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4ltq9"
Sep 08 15:09:14.572051 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:14.571020    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd64afe66_e605_40e6_a64f_11dd8f2165cf.slice/crio-3ef97b70b0df1fa6a7ad9a5c11fd0406203d46fe448636b4d1bddfe35589aef5 WatchSource:0}: Error finding container 3ef97b70b0df1fa6a7ad9a5c11fd0406203d46fe448636b4d1bddfe35589aef5: Status 404 returned error can't find the container with id 3ef97b70b0df1fa6a7ad9a5c11fd0406203d46fe448636b4d1bddfe35589aef5
Sep 08 15:09:14.577478 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.577460    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qvxxd"
Sep 08 15:09:14.579703 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:14.579668    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd33f854f_48e2_45ca_8289_0f5cb0817be7.slice/crio-0675c741388e715b65003809c0c168720bec25b3cb54c6d69fddd95d6dfd0bca WatchSource:0}: Error finding container 0675c741388e715b65003809c0c168720bec25b3cb54c6d69fddd95d6dfd0bca: Status 404 returned error can't find the container with id 0675c741388e715b65003809c0c168720bec25b3cb54c6d69fddd95d6dfd0bca
Sep 08 15:09:14.582615 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.582594    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vx9n2"
Sep 08 15:09:14.585738 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:14.585716    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddea9369e_6c20_4b59_a04b_eb20d2f8cca5.slice/crio-8a754d45d643fbfad5718582faada4e9c88b1811f1874f0938984967000f1bf3 WatchSource:0}: Error finding container 8a754d45d643fbfad5718582faada4e9c88b1811f1874f0938984967000f1bf3: Status 404 returned error can't find the container with id 8a754d45d643fbfad5718582faada4e9c88b1811f1874f0938984967000f1bf3
Sep 08 15:09:14.587854 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.587835    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fchvk"
Sep 08 15:09:14.591151 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:14.591124    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb368e23_9846_425c_be55_cee17f483954.slice/crio-e9d3c2e31fbf0e0c319711c48d9b9e08176bb1f8bafd3263698672aedccd1a9b WatchSource:0}: Error finding container e9d3c2e31fbf0e0c319711c48d9b9e08176bb1f8bafd3263698672aedccd1a9b: Status 404 returned error can't find the container with id e9d3c2e31fbf0e0c319711c48d9b9e08176bb1f8bafd3263698672aedccd1a9b
Sep 08 15:09:14.594268 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.594083    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6"
Sep 08 15:09:14.596483 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:14.596455    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9b25500_9302_461f_9458_f2e4a5dfbcd7.slice/crio-8fc309074e3c72d305a2e8df8b9aee91b704edb920b55040daf2e64004be0682 WatchSource:0}: Error finding container 8fc309074e3c72d305a2e8df8b9aee91b704edb920b55040daf2e64004be0682: Status 404 returned error can't find the container with id 8fc309074e3c72d305a2e8df8b9aee91b704edb920b55040daf2e64004be0682
Sep 08 15:09:14.600923 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.600641    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-727ns"
Sep 08 15:09:14.602530 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:14.602278    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b986d3_6372_4117_94d8_7f26bcc542da.slice/crio-6e2c3257495dd8d51166483ed7740a525700b92f2e18925dd3b00b14ed08cde9 WatchSource:0}: Error finding container 6e2c3257495dd8d51166483ed7740a525700b92f2e18925dd3b00b14ed08cde9: Status 404 returned error can't find the container with id 6e2c3257495dd8d51166483ed7740a525700b92f2e18925dd3b00b14ed08cde9
Sep 08 15:09:14.606125 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.606107    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zrlxq"
Sep 08 15:09:14.610241 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.610221    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:14.610900 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:14.610882    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0011d520_9d6d_4932_91f9_80084021b951.slice/crio-f2e8fbeafe7d1557dd30f1203e93d20bf18aacc33a2fca1df14d939fd0bb6270 WatchSource:0}: Error finding container f2e8fbeafe7d1557dd30f1203e93d20bf18aacc33a2fca1df14d939fd0bb6270: Status 404 returned error can't find the container with id f2e8fbeafe7d1557dd30f1203e93d20bf18aacc33a2fca1df14d939fd0bb6270
Sep 08 15:09:14.621641 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:14.621620    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6586e3c3_662b_42dd_9891_3b6c94042172.slice/crio-1e804193d51de8d200cc41ff9b1cd76af14a9b55ab7391e9fb945b3b12f86805 WatchSource:0}: Error finding container 1e804193d51de8d200cc41ff9b1cd76af14a9b55ab7391e9fb945b3b12f86805: Status 404 returned error can't find the container with id 1e804193d51de8d200cc41ff9b1cd76af14a9b55ab7391e9fb945b3b12f86805
Sep 08 15:09:14.876103 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.876040    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:14.876103 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:14.876076    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:14.876262 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.876172    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:09:14.876262 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.876173    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:09:14.876262 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.876218    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret podName:18237a47-8e2c-42f5-b960-f56188420981 nodeName:}" failed. No retries permitted until 2026-09-08 15:09:15.876205069 +0000 UTC m=+3.962155088 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret") pod "global-pull-secret-syncer-lbq79" (UID: "18237a47-8e2c-42f5-b960-f56188420981") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:09:14.876262 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:14.876231    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs podName:4f348729-57f1-40ab-ae97-85bfefa7d80e nodeName:}" failed. No retries permitted until 2026-09-08 15:09:15.876225565 +0000 UTC m=+3.962175585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs") pod "network-metrics-daemon-9vwnd" (UID: "4f348729-57f1-40ab-ae97-85bfefa7d80e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:09:15.077319 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.077289    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rxx\" (UniqueName: \"kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx\") pod \"network-check-target-dlpvm\" (UID: \"7ccbb705-396e-4f0f-8756-b1a81ae1a840\") " pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:15.077526 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:15.077505    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:09:15.077594 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:15.077532    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:09:15.077594 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:15.077546    2575 projected.go:196] Error preparing data for projected volume kube-api-access-g4rxx for pod openshift-network-diagnostics/network-check-target-dlpvm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:09:15.077661 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:15.077608    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx podName:7ccbb705-396e-4f0f-8756-b1a81ae1a840 nodeName:}" failed. No retries permitted until 2026-09-08 15:09:16.077587529 +0000 UTC m=+4.163537556 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4rxx" (UniqueName: "kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx") pod "network-check-target-dlpvm" (UID: "7ccbb705-396e-4f0f-8756-b1a81ae1a840") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:09:15.308248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.308172    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 15:04:13 +0000 UTC" deadline="2028-03-27 19:14:26.492175073 +0000 UTC"
Sep 08 15:09:15.308248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.308210    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13588h5m11.183967572s"
Sep 08 15:09:15.426321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.425837    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:15.426321 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:15.425958    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:15.437271 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.436987    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc" event={"ID":"6586e3c3-662b-42dd-9891-3b6c94042172","Type":"ContainerStarted","Data":"1e804193d51de8d200cc41ff9b1cd76af14a9b55ab7391e9fb945b3b12f86805"}
Sep 08 15:09:15.438932 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.438899    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6" event={"ID":"75b986d3-6372-4117-94d8-7f26bcc542da","Type":"ContainerStarted","Data":"6e2c3257495dd8d51166483ed7740a525700b92f2e18925dd3b00b14ed08cde9"}
Sep 08 15:09:15.440531 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.440505    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vx9n2" event={"ID":"cb368e23-9846-425c-be55-cee17f483954","Type":"ContainerStarted","Data":"e9d3c2e31fbf0e0c319711c48d9b9e08176bb1f8bafd3263698672aedccd1a9b"}
Sep 08 15:09:15.442747 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.442719    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qvxxd" event={"ID":"dea9369e-6c20-4b59-a04b-eb20d2f8cca5","Type":"ContainerStarted","Data":"8a754d45d643fbfad5718582faada4e9c88b1811f1874f0938984967000f1bf3"}
Sep 08 15:09:15.444398 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.444371    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zrlxq" event={"ID":"e13f4f32-5641-4b50-8b14-e94551c552c0","Type":"ContainerStarted","Data":"785162a21ae545a9045078dcab1538df810f66337a774f451f764e39b1af81de"}
Sep 08 15:09:15.445764 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.445738    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-727ns" event={"ID":"0011d520-9d6d-4932-91f9-80084021b951","Type":"ContainerStarted","Data":"f2e8fbeafe7d1557dd30f1203e93d20bf18aacc33a2fca1df14d939fd0bb6270"}
Sep 08 15:09:15.447894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.447873    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fchvk" event={"ID":"e9b25500-9302-461f-9458-f2e4a5dfbcd7","Type":"ContainerStarted","Data":"8fc309074e3c72d305a2e8df8b9aee91b704edb920b55040daf2e64004be0682"}
Sep 08 15:09:15.449540 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.449517    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4ltq9" event={"ID":"d33f854f-48e2-45ca-8289-0f5cb0817be7","Type":"ContainerStarted","Data":"0675c741388e715b65003809c0c168720bec25b3cb54c6d69fddd95d6dfd0bca"}
Sep 08 15:09:15.450983 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.450948    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8fh7x" event={"ID":"d64afe66-e605-40e6-a64f-11dd8f2165cf","Type":"ContainerStarted","Data":"3ef97b70b0df1fa6a7ad9a5c11fd0406203d46fe448636b4d1bddfe35589aef5"}
Sep 08 15:09:15.883284 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.882535    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:15.883284 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:15.882589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:15.883284 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:15.882752    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:09:15.883284 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:15.882812    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret podName:18237a47-8e2c-42f5-b960-f56188420981 nodeName:}" failed. No retries permitted until 2026-09-08 15:09:17.882793863 +0000 UTC m=+5.968743883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret") pod "global-pull-secret-syncer-lbq79" (UID: "18237a47-8e2c-42f5-b960-f56188420981") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:09:15.883284 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:15.883196    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:09:15.883284 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:15.883247    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs podName:4f348729-57f1-40ab-ae97-85bfefa7d80e nodeName:}" failed. No retries permitted until 2026-09-08 15:09:17.88323058 +0000 UTC m=+5.969180617 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs") pod "network-metrics-daemon-9vwnd" (UID: "4f348729-57f1-40ab-ae97-85bfefa7d80e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:09:16.085164 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:16.084742    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rxx\" (UniqueName: \"kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx\") pod \"network-check-target-dlpvm\" (UID: \"7ccbb705-396e-4f0f-8756-b1a81ae1a840\") " pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:16.085164 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:16.084890    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:09:16.085164 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:16.084908    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:09:16.085164 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:16.084920    2575 projected.go:196] Error preparing data for projected volume kube-api-access-g4rxx for pod openshift-network-diagnostics/network-check-target-dlpvm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:09:16.085164 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:16.084975    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx podName:7ccbb705-396e-4f0f-8756-b1a81ae1a840 nodeName:}" failed. No retries permitted until 2026-09-08 15:09:18.084958829 +0000 UTC m=+6.170908856 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4rxx" (UniqueName: "kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx") pod "network-check-target-dlpvm" (UID: "7ccbb705-396e-4f0f-8756-b1a81ae1a840") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:09:16.428863 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:16.428292    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:16.428863 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:16.428423    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:16.428863 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:16.428530    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:16.428863 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:16.428626    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:16.456557 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:16.456526    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-76.ec2.internal" event={"ID":"b674e83ab0483a337814cf332a4de8c1","Type":"ContainerStarted","Data":"18c67c600db3128d9cc19acaaedfef7a7201e033b11c9adc5b40641c57b2e468"}
Sep 08 15:09:16.474865 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:16.474815    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-76.ec2.internal" podStartSLOduration=3.4747995400000002 podStartE2EDuration="3.47479954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:09:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:16.47437509 +0000 UTC m=+4.560325146" watchObservedRunningTime="2026-09-08 15:09:16.47479954 +0000 UTC m=+4.560749585"
Sep 08 15:09:17.425317 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:17.425290    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:17.425444 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:17.425411    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:17.898449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:17.898337    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:17.898449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:17.898392    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:17.899070 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:17.898479    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:09:17.899070 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:17.898558    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret podName:18237a47-8e2c-42f5-b960-f56188420981 nodeName:}" failed. No retries permitted until 2026-09-08 15:09:21.898538168 +0000 UTC m=+9.984488202 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret") pod "global-pull-secret-syncer-lbq79" (UID: "18237a47-8e2c-42f5-b960-f56188420981") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:09:17.899070 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:17.898479    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:09:17.899070 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:17.898616    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs podName:4f348729-57f1-40ab-ae97-85bfefa7d80e nodeName:}" failed. No retries permitted until 2026-09-08 15:09:21.898602797 +0000 UTC m=+9.984552826 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs") pod "network-metrics-daemon-9vwnd" (UID: "4f348729-57f1-40ab-ae97-85bfefa7d80e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:09:18.100045 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:18.100011    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rxx\" (UniqueName: \"kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx\") pod \"network-check-target-dlpvm\" (UID: \"7ccbb705-396e-4f0f-8756-b1a81ae1a840\") " pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:18.100196 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:18.100144    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:09:18.100196 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:18.100171    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:09:18.100196 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:18.100186    2575 projected.go:196] Error preparing data for projected volume kube-api-access-g4rxx for pod openshift-network-diagnostics/network-check-target-dlpvm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:09:18.100362 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:18.100246    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx podName:7ccbb705-396e-4f0f-8756-b1a81ae1a840 nodeName:}" failed. No retries permitted until 2026-09-08 15:09:22.100228225 +0000 UTC m=+10.186178249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4rxx" (UniqueName: "kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx") pod "network-check-target-dlpvm" (UID: "7ccbb705-396e-4f0f-8756-b1a81ae1a840") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:09:18.426333 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:18.426302    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:18.426501 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:18.426428    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:18.426894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:18.426875    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:18.427012 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:18.426992    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:19.426012 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:19.425979    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:19.426447 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:19.426103    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:20.426170 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:20.426134    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:20.426170 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:20.426170    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:20.426686 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:20.426265    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:20.426686 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:20.426359    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:21.425612 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:21.425537    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:21.425768 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:21.425675    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:21.933486 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:21.933368    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:21.933486 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:21.933429    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:21.933943 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:21.933552    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:09:21.933943 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:21.933629    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret podName:18237a47-8e2c-42f5-b960-f56188420981 nodeName:}" failed. No retries permitted until 2026-09-08 15:09:29.933609722 +0000 UTC m=+18.019559755 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret") pod "global-pull-secret-syncer-lbq79" (UID: "18237a47-8e2c-42f5-b960-f56188420981") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:09:21.933943 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:21.933552    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:09:21.933943 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:21.933852    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs podName:4f348729-57f1-40ab-ae97-85bfefa7d80e nodeName:}" failed. No retries permitted until 2026-09-08 15:09:29.933831395 +0000 UTC m=+18.019781420 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs") pod "network-metrics-daemon-9vwnd" (UID: "4f348729-57f1-40ab-ae97-85bfefa7d80e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:09:22.135675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:22.135386    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rxx\" (UniqueName: \"kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx\") pod \"network-check-target-dlpvm\" (UID: \"7ccbb705-396e-4f0f-8756-b1a81ae1a840\") " pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:22.135675 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:22.135613    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:09:22.135675 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:22.135634    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:09:22.135675 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:22.135647    2575 projected.go:196] Error preparing data for projected volume kube-api-access-g4rxx for pod openshift-network-diagnostics/network-check-target-dlpvm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:09:22.135977 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:22.135708    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx podName:7ccbb705-396e-4f0f-8756-b1a81ae1a840 nodeName:}" failed. No retries permitted until 2026-09-08 15:09:30.135690079 +0000 UTC m=+18.221640100 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4rxx" (UniqueName: "kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx") pod "network-check-target-dlpvm" (UID: "7ccbb705-396e-4f0f-8756-b1a81ae1a840") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:09:22.427656 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:22.427029    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:22.427656 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:22.427131    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:22.427656 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:22.427512    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:22.427656 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:22.427611    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:23.425965 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:23.425883    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:23.426634 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:23.426006    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:24.429465 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:24.429429    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:24.429465 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:24.429452    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:24.429951 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:24.429592    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:24.429951 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:24.429701    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:25.425504 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:25.425413    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:25.425646 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:25.425547    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:26.428902 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:26.428876    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:26.429341 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:26.428876    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:26.429341 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:26.428978    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:26.429341 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:26.429065    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:27.426063 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:27.426028    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:27.426248 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:27.426153    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:28.428276 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:28.428243    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:28.428720 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:28.428364    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:28.428720 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:28.428427    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:28.428720 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:28.428557    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:29.425968 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:29.425938    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:29.426163 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:29.426040    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:29.989395 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:29.989355    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:29.989836 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:29.989417    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:29.989836 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:29.989467    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:09:29.989836 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:29.989539    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:09:29.989836 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:29.989549    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs podName:4f348729-57f1-40ab-ae97-85bfefa7d80e nodeName:}" failed. No retries permitted until 2026-09-08 15:09:45.989533327 +0000 UTC m=+34.075483366 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs") pod "network-metrics-daemon-9vwnd" (UID: "4f348729-57f1-40ab-ae97-85bfefa7d80e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:09:29.989836 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:29.989583    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret podName:18237a47-8e2c-42f5-b960-f56188420981 nodeName:}" failed. No retries permitted until 2026-09-08 15:09:45.989567585 +0000 UTC m=+34.075517621 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret") pod "global-pull-secret-syncer-lbq79" (UID: "18237a47-8e2c-42f5-b960-f56188420981") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:09:30.190981 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:30.190948    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rxx\" (UniqueName: \"kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx\") pod \"network-check-target-dlpvm\" (UID: \"7ccbb705-396e-4f0f-8756-b1a81ae1a840\") " pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:30.191133 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:30.191090    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:09:30.191133 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:30.191110    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:09:30.191133 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:30.191120    2575 projected.go:196] Error preparing data for projected volume kube-api-access-g4rxx for pod openshift-network-diagnostics/network-check-target-dlpvm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:09:30.191309 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:30.191186    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx podName:7ccbb705-396e-4f0f-8756-b1a81ae1a840 nodeName:}" failed. No retries permitted until 2026-09-08 15:09:46.191168032 +0000 UTC m=+34.277118077 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4rxx" (UniqueName: "kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx") pod "network-check-target-dlpvm" (UID: "7ccbb705-396e-4f0f-8756-b1a81ae1a840") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:09:30.425871 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:30.425800    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:30.426037 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:30.425800    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:30.426037 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:30.425914    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:30.426154 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:30.426048    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:31.425669 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:31.425638    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:31.426144 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:31.425751    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:32.425956 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:32.425928    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:32.426346 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:32.426048    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:32.426346 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:32.426084    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:32.426346 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:32.426145    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:33.426362 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.426073    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:33.426855 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:33.426442    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:33.491009 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.490977    2575 generic.go:356] "Generic (PLEG): container finished" podID="87be9da9c6492bc0bdf474fd1bdc742a" containerID="416174de3e34424f0ad06e9801407da41959cea64d04c83eea2ead69448f4631" exitCode=0
Sep 08 15:09:33.491114 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.491067    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal" event={"ID":"87be9da9c6492bc0bdf474fd1bdc742a","Type":"ContainerDied","Data":"416174de3e34424f0ad06e9801407da41959cea64d04c83eea2ead69448f4631"}
Sep 08 15:09:33.493261 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.493104    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zrlxq" event={"ID":"e13f4f32-5641-4b50-8b14-e94551c552c0","Type":"ContainerStarted","Data":"d70f09d488d9f1fbcf3b27283b5212bf61be129921b1f9b8a3ba73db0ce727c5"}
Sep 08 15:09:33.494958 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.494919    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-727ns" event={"ID":"0011d520-9d6d-4932-91f9-80084021b951","Type":"ContainerStarted","Data":"977ad3192c926f3511679f042d8746b6ad478346b7f049813f7d8af3c34f3d67"}
Sep 08 15:09:33.496695 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.496672    2575 generic.go:356] "Generic (PLEG): container finished" podID="e9b25500-9302-461f-9458-f2e4a5dfbcd7" containerID="4ca77f0ea13aa96cae391a55d09b6124683ebde9d17992a993d1b4a4614bd62f" exitCode=0
Sep 08 15:09:33.496790 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.496755    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fchvk" event={"ID":"e9b25500-9302-461f-9458-f2e4a5dfbcd7","Type":"ContainerDied","Data":"4ca77f0ea13aa96cae391a55d09b6124683ebde9d17992a993d1b4a4614bd62f"}
Sep 08 15:09:33.498575 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.498552    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4ltq9" event={"ID":"d33f854f-48e2-45ca-8289-0f5cb0817be7","Type":"ContainerStarted","Data":"07f6bfccb50f5df4a5f1a3250dd76570bc94efc511051f0ff5f945613cd5ff31"}
Sep 08 15:09:33.500462 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.500439    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8fh7x" event={"ID":"d64afe66-e605-40e6-a64f-11dd8f2165cf","Type":"ContainerStarted","Data":"3e124d79cfcb9da4c6af115d244286cd68cac614d08fa6b1a523fe57e8c6b918"}
Sep 08 15:09:33.504117 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.504094    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc" event={"ID":"6586e3c3-662b-42dd-9891-3b6c94042172","Type":"ContainerStarted","Data":"b17b8ba89a655652bf4bc1b8b13834ca57053a38ddc99e03baadff364f0036bf"}
Sep 08 15:09:33.504215 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.504168    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc" event={"ID":"6586e3c3-662b-42dd-9891-3b6c94042172","Type":"ContainerStarted","Data":"45cc9393705e7eb87e30c4ce1918aec83323641b928b1de973e369ff10c57afc"}
Sep 08 15:09:33.504276 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.504220    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc" event={"ID":"6586e3c3-662b-42dd-9891-3b6c94042172","Type":"ContainerStarted","Data":"fab93f253bd093524dc73030629aacbbdf375bbe7d63d8d826f130a5dfdc29c9"}
Sep 08 15:09:33.504334 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.504267    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc" event={"ID":"6586e3c3-662b-42dd-9891-3b6c94042172","Type":"ContainerStarted","Data":"bfe2ebee21f97a1cdcb88758e94ed8149d7d0b5c975f96963129520a259c9faf"}
Sep 08 15:09:33.504334 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.504322    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc" event={"ID":"6586e3c3-662b-42dd-9891-3b6c94042172","Type":"ContainerStarted","Data":"6dc32f3b25b92968e6c9a9afff1cf2fe662550d572b18579608ca82635defe7e"}
Sep 08 15:09:33.504440 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.504370    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc" event={"ID":"6586e3c3-662b-42dd-9891-3b6c94042172","Type":"ContainerStarted","Data":"9e8de193156a9990c4f306974d076d9de5316cff276bd7f1b49f15126b900f86"}
Sep 08 15:09:33.505828 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.505810    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6" event={"ID":"75b986d3-6372-4117-94d8-7f26bcc542da","Type":"ContainerStarted","Data":"650ffc84d93ec07ae3189b81aa3ff03fa3d4b30d8a724069efa6cb6e21322397"}
Sep 08 15:09:33.507391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.507367    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qvxxd" event={"ID":"dea9369e-6c20-4b59-a04b-eb20d2f8cca5","Type":"ContainerStarted","Data":"427fe9115030b82e660f7dc67a3bb6d3ba2e4487b2c00e7a250e9f01bd3fc429"}
Sep 08 15:09:33.543807 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.543762    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8fh7x" podStartSLOduration=3.5661036409999998 podStartE2EDuration="21.543747541s" totalImagesPullingTime="17.9776439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:33.54315497 +0000 UTC m=+21.629105013" watchObservedRunningTime="2026-09-08 15:09:33.543747541 +0000 UTC m=+21.629697583"
Sep 08 15:09:33.569230 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.569183    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qvxxd" podStartSLOduration=11.155151202 podStartE2EDuration="21.569167414s" totalImagesPullingTime="10.414016212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:33.568787737 +0000 UTC m=+21.654737782" watchObservedRunningTime="2026-09-08 15:09:33.569167414 +0000 UTC m=+21.655117458"
Sep 08 15:09:33.591244 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.590896    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zrlxq" podStartSLOduration=3.580178384 podStartE2EDuration="21.590884224s" totalImagesPullingTime="18.01070584s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:33.590849048 +0000 UTC m=+21.676799088" watchObservedRunningTime="2026-09-08 15:09:33.590884224 +0000 UTC m=+21.676834267"
Sep 08 15:09:33.612422 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.612376    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4ltq9" podStartSLOduration=11.192875717 podStartE2EDuration="21.612359417s" totalImagesPullingTime="10.4194837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:33.612355249 +0000 UTC m=+21.698305292" watchObservedRunningTime="2026-09-08 15:09:33.612359417 +0000 UTC m=+21.698309462"
Sep 08 15:09:33.685506 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:33.685436    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-727ns" podStartSLOduration=3.723460352 podStartE2EDuration="21.685416104s" totalImagesPullingTime="17.961955752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:33.684821375 +0000 UTC m=+21.770771412" watchObservedRunningTime="2026-09-08 15:09:33.685416104 +0000 UTC m=+21.771366148"
Sep 08 15:09:34.007720 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.007695    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 08 15:09:34.415784 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.415758    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8fh7x"
Sep 08 15:09:34.416358 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.416339    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8fh7x"
Sep 08 15:09:34.425329 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.425309    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:34.425475 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:34.425451    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:34.425551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.425478    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:34.425614 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:34.425595    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:34.461967 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.461829    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T15:09:34.007713468Z","UUID":"cd71339d-6c12-41e5-a0f4-0a6cb4736a82","Handler":null,"Name":"","Endpoint":""}
Sep 08 15:09:34.463368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.463346    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 08 15:09:34.463478 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.463381    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 08 15:09:34.511193 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.511160    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal" event={"ID":"87be9da9c6492bc0bdf474fd1bdc742a","Type":"ContainerStarted","Data":"a0df631fc151a81bea5f5390507b45232445fa9c117f18832eef8a5235608be5"}
Sep 08 15:09:34.513200 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.513170    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6" event={"ID":"75b986d3-6372-4117-94d8-7f26bcc542da","Type":"ContainerStarted","Data":"0d9c5717c33165fa967a7a75fde501b7bba413acc55468ac8c66d843ed9e2d15"}
Sep 08 15:09:34.514656 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.514619    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vx9n2" event={"ID":"cb368e23-9846-425c-be55-cee17f483954","Type":"ContainerStarted","Data":"a5618dfa203bcdf1144bca5256c2bc81e72729ba96f5205566c63bc756612d09"}
Sep 08 15:09:34.515445 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.515425    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8fh7x"
Sep 08 15:09:34.515559 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.515521    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8fh7x"
Sep 08 15:09:34.529856 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.529815    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-76.ec2.internal" podStartSLOduration=21.529798721 podStartE2EDuration="21.529798721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:09:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:34.529709961 +0000 UTC m=+22.615660004" watchObservedRunningTime="2026-09-08 15:09:34.529798721 +0000 UTC m=+22.615748764"
Sep 08 15:09:34.566300 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.566266    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vx9n2" podStartSLOduration=4.6080654 podStartE2EDuration="22.566222152s" totalImagesPullingTime="17.958156752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:34.566136336 +0000 UTC m=+22.652086382" watchObservedRunningTime="2026-09-08 15:09:34.566222152 +0000 UTC m=+22.652172194"
Sep 08 15:09:34.754056 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.753979    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:34.871476 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:34.871445    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:35.425963 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:35.425798    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:35.426094 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:35.426030    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:35.520821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:35.520788    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc" event={"ID":"6586e3c3-662b-42dd-9891-3b6c94042172","Type":"ContainerStarted","Data":"2d365316e79b32d51fcd72639a282bcae93bddb8377546a94fea42de1a1d4e90"}
Sep 08 15:09:35.523056 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:35.523021    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6" event={"ID":"75b986d3-6372-4117-94d8-7f26bcc542da","Type":"ContainerStarted","Data":"7c597d25449913571a1c41b441b617db4a3a31c8183a3d117a1d6208eee33eb4"}
Sep 08 15:09:35.546646 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:35.544569    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bwkj6" podStartSLOduration=3.651647451 podStartE2EDuration="23.544540766s" totalImagesPullingTime="19.892893315s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:09:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:35.543745991 +0000 UTC m=+23.629696034" watchObservedRunningTime="2026-09-08 15:09:35.544540766 +0000 UTC m=+23.630490808"
Sep 08 15:09:36.425697 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:36.425663    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:36.425962 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:36.425788    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:36.425962 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:36.425844    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:36.426077 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:36.425977    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:36.713349 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:36.713274    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:36.831379 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:36.831348    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:36.931433 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:36.931398    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:37.425946 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:37.425658    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:37.425946 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:37.425782    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:37.532363 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:37.532321    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc" event={"ID":"6586e3c3-662b-42dd-9891-3b6c94042172","Type":"ContainerStarted","Data":"6e7604a0809337b81d4edb986e618d9a26bb436ae76551f782dc44889f6b289d"}
Sep 08 15:09:37.532720 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:37.532699    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:37.549927 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:37.549902    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:37.570871 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:37.570782    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc" podStartSLOduration=7.594415833 podStartE2EDuration="25.570764024s" totalImagesPullingTime="17.976348191s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:37.56984742 +0000 UTC m=+25.655797474" watchObservedRunningTime="2026-09-08 15:09:37.570764024 +0000 UTC m=+25.656714067"
Sep 08 15:09:38.425979 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:38.425782    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:38.426424 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:38.425785    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:38.426424 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:38.426089    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:38.426424 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:38.426160    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:38.534592 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:38.534563    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:38.534771 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:38.534675    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:38.551575 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:38.551384    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:09:39.368583 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:39.368539    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lbq79"]
Sep 08 15:09:39.368852 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:39.368674    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:39.368852 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:39.368775    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:39.368852 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:39.368819    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dlpvm"]
Sep 08 15:09:39.369044 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:39.368904    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:39.369044 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:39.368974    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:39.370182 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:39.370153    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9vwnd"]
Sep 08 15:09:39.370363 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:39.370276    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:39.370436 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:39.370417    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:40.102334 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:40.102276    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:40.539673 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:40.539642    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:40.539813 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:40.539790    2575 generic.go:356] "Generic (PLEG): container finished" podID="e9b25500-9302-461f-9458-f2e4a5dfbcd7" containerID="bac08e547dd71a6b7d92d65f5a33ebc0af226444c94b394c871de46fa5506f7a" exitCode=0
Sep 08 15:09:40.539871 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:40.539826    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fchvk" event={"ID":"e9b25500-9302-461f-9458-f2e4a5dfbcd7","Type":"ContainerDied","Data":"bac08e547dd71a6b7d92d65f5a33ebc0af226444c94b394c871de46fa5506f7a"}
Sep 08 15:09:40.610873 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:40.610848    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:41.027929 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:41.027891    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:41.133430 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:41.133400    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:41.425335 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:41.425309    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:41.425452 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:41.425410    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:41.425525 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:41.425407    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:41.425575 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:41.425524    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:41.425626 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:41.425408    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:41.425686 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:41.425619    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:42.546033 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:42.545861    2575 generic.go:356] "Generic (PLEG): container finished" podID="e9b25500-9302-461f-9458-f2e4a5dfbcd7" containerID="f8caab1f10cf395face76240d81fc2cd70d17533a2766aa4313cb15bee443dbd" exitCode=0
Sep 08 15:09:42.546367 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:42.545945    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fchvk" event={"ID":"e9b25500-9302-461f-9458-f2e4a5dfbcd7","Type":"ContainerDied","Data":"f8caab1f10cf395face76240d81fc2cd70d17533a2766aa4313cb15bee443dbd"}
Sep 08 15:09:43.425774 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:43.425747    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:43.425857 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:43.425778    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:43.425857 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:43.425837    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dlpvm" podUID="7ccbb705-396e-4f0f-8756-b1a81ae1a840"
Sep 08 15:09:43.425918 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:43.425876    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:43.425981 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:43.425965    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:09:43.426046 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:43.426032    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lbq79" podUID="18237a47-8e2c-42f5-b960-f56188420981"
Sep 08 15:09:43.553449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:43.553384    2575 generic.go:356] "Generic (PLEG): container finished" podID="e9b25500-9302-461f-9458-f2e4a5dfbcd7" containerID="9b2e2dada256a457663ea2f5a5f11b89b761809000d6d99c475f0b8b3e15b088" exitCode=0
Sep 08 15:09:43.553449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:43.553426    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fchvk" event={"ID":"e9b25500-9302-461f-9458-f2e4a5dfbcd7","Type":"ContainerDied","Data":"9b2e2dada256a457663ea2f5a5f11b89b761809000d6d99c475f0b8b3e15b088"}
Sep 08 15:09:45.209822 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.209791    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-76.ec2.internal" event="NodeReady"
Sep 08 15:09:45.210244 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.209927    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 08 15:09:45.276306 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.276280    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nvc47"]
Sep 08 15:09:45.329194 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.329170    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:45.332180 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.332161    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-p4szr"]
Sep 08 15:09:45.332311 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.332293    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nvc47"
Sep 08 15:09:45.335442 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.335421    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 08 15:09:45.335613 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.335479    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 08 15:09:45.335738 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.335723    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9rn4v\""
Sep 08 15:09:45.335808 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.335752    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 08 15:09:45.346596 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.346547    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nvc47"]
Sep 08 15:09:45.346596 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.346574    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p4szr"]
Sep 08 15:09:45.346733 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.346642    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:45.349261 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.349242    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 08 15:09:45.349681 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.349660    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cc4mw\""
Sep 08 15:09:45.349787 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.349665    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 08 15:09:45.405195 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.405174    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96p6s\" (UniqueName: \"kubernetes.io/projected/a2e96079-8d75-4cd8-b77a-49f86c67a787-kube-api-access-96p6s\") pod \"ingress-canary-nvc47\" (UID: \"a2e96079-8d75-4cd8-b77a-49f86c67a787\") " pod="openshift-ingress-canary/ingress-canary-nvc47"
Sep 08 15:09:45.405300 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.405213    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2e96079-8d75-4cd8-b77a-49f86c67a787-cert\") pod \"ingress-canary-nvc47\" (UID: \"a2e96079-8d75-4cd8-b77a-49f86c67a787\") " pod="openshift-ingress-canary/ingress-canary-nvc47"
Sep 08 15:09:45.408531 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.408484    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:45.425300 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.425276    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:45.425300 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.425285    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:45.425709 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.425502    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:45.428849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.428827    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 15:09:45.429438 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.429156    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 15:09:45.429438 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.429158    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 15:09:45.429438 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.429190    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t2q2k\""
Sep 08 15:09:45.429438 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.429297    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lf7xf\""
Sep 08 15:09:45.429438 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.429342    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 15:09:45.506291 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.506265    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96p6s\" (UniqueName: \"kubernetes.io/projected/a2e96079-8d75-4cd8-b77a-49f86c67a787-kube-api-access-96p6s\") pod \"ingress-canary-nvc47\" (UID: \"a2e96079-8d75-4cd8-b77a-49f86c67a787\") " pod="openshift-ingress-canary/ingress-canary-nvc47"
Sep 08 15:09:45.506399 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.506314    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2e96079-8d75-4cd8-b77a-49f86c67a787-cert\") pod \"ingress-canary-nvc47\" (UID: \"a2e96079-8d75-4cd8-b77a-49f86c67a787\") " pod="openshift-ingress-canary/ingress-canary-nvc47"
Sep 08 15:09:45.506399 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.506351    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-config-volume\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:45.506527 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.506404    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-tmp-dir\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:45.506527 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.506435    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsc2t\" (UniqueName: \"kubernetes.io/projected/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-kube-api-access-fsc2t\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:45.506622 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.506506    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:45.511133 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.511104    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2e96079-8d75-4cd8-b77a-49f86c67a787-cert\") pod \"ingress-canary-nvc47\" (UID: \"a2e96079-8d75-4cd8-b77a-49f86c67a787\") " pod="openshift-ingress-canary/ingress-canary-nvc47"
Sep 08 15:09:45.516535 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.516514    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96p6s\" (UniqueName: \"kubernetes.io/projected/a2e96079-8d75-4cd8-b77a-49f86c67a787-kube-api-access-96p6s\") pod \"ingress-canary-nvc47\" (UID: \"a2e96079-8d75-4cd8-b77a-49f86c67a787\") " pod="openshift-ingress-canary/ingress-canary-nvc47"
Sep 08 15:09:45.607066 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.607043    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-tmp-dir\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:45.607181 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.607076    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsc2t\" (UniqueName: \"kubernetes.io/projected/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-kube-api-access-fsc2t\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:45.607181 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.607098    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:45.607284 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.607197    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-config-volume\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:45.607335 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:45.607288    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:09:45.607387 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:45.607365    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls podName:f621c1ae-a9a4-4c7d-979c-adfbcafaf33a nodeName:}" failed. No retries permitted until 2026-09-08 15:09:46.107345426 +0000 UTC m=+34.193295475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls") pod "dns-default-p4szr" (UID: "f621c1ae-a9a4-4c7d-979c-adfbcafaf33a") : secret "dns-default-metrics-tls" not found
Sep 08 15:09:45.607445 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.607386    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-tmp-dir\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:45.607769 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.607746    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-config-volume\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:45.616343 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.616325    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsc2t\" (UniqueName: \"kubernetes.io/projected/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-kube-api-access-fsc2t\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:45.644318 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.644295    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nvc47"
Sep 08 15:09:45.966748 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:45.966680    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nvc47"]
Sep 08 15:09:45.971246 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:45.971217    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2e96079_8d75_4cd8_b77a_49f86c67a787.slice/crio-62add2157cfb94a1bfc34d5e03a714d0877c53a4c26b27a9f8bc290e040fc376 WatchSource:0}: Error finding container 62add2157cfb94a1bfc34d5e03a714d0877c53a4c26b27a9f8bc290e040fc376: Status 404 returned error can't find the container with id 62add2157cfb94a1bfc34d5e03a714d0877c53a4c26b27a9f8bc290e040fc376
Sep 08 15:09:46.010307 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:46.010283    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:09:46.010422 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:46.010330    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:46.010422 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:46.010453    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 08 15:09:46.010684 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:46.010545    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs podName:4f348729-57f1-40ab-ae97-85bfefa7d80e nodeName:}" failed. No retries permitted until 2026-09-08 15:10:18.010524393 +0000 UTC m=+66.096474430 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs") pod "network-metrics-daemon-9vwnd" (UID: "4f348729-57f1-40ab-ae97-85bfefa7d80e") : secret "metrics-daemon-secret" not found
Sep 08 15:09:46.013283 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:46.013254    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/18237a47-8e2c-42f5-b960-f56188420981-original-pull-secret\") pod \"global-pull-secret-syncer-lbq79\" (UID: \"18237a47-8e2c-42f5-b960-f56188420981\") " pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:46.036102 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:46.036083    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lbq79"
Sep 08 15:09:46.111517 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:46.111419    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:46.111650 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:46.111619    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:09:46.111831 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:46.111815    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls podName:f621c1ae-a9a4-4c7d-979c-adfbcafaf33a nodeName:}" failed. No retries permitted until 2026-09-08 15:09:47.111784628 +0000 UTC m=+35.197734691 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls") pod "dns-default-p4szr" (UID: "f621c1ae-a9a4-4c7d-979c-adfbcafaf33a") : secret "dns-default-metrics-tls" not found
Sep 08 15:09:46.212433 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:46.212404    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rxx\" (UniqueName: \"kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx\") pod \"network-check-target-dlpvm\" (UID: \"7ccbb705-396e-4f0f-8756-b1a81ae1a840\") " pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:46.215606 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:46.215558    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4rxx\" (UniqueName: \"kubernetes.io/projected/7ccbb705-396e-4f0f-8756-b1a81ae1a840-kube-api-access-g4rxx\") pod \"network-check-target-dlpvm\" (UID: \"7ccbb705-396e-4f0f-8756-b1a81ae1a840\") " pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:46.343366 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:46.343304    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lbq79"]
Sep 08 15:09:46.348250 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:46.348230    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:46.562432 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:46.562402    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nvc47" event={"ID":"a2e96079-8d75-4cd8-b77a-49f86c67a787","Type":"ContainerStarted","Data":"62add2157cfb94a1bfc34d5e03a714d0877c53a4c26b27a9f8bc290e040fc376"}
Sep 08 15:09:47.119714 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:47.119679    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:47.119903 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:47.119846    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:09:47.119976 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:47.119921    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls podName:f621c1ae-a9a4-4c7d-979c-adfbcafaf33a nodeName:}" failed. No retries permitted until 2026-09-08 15:09:49.119901757 +0000 UTC m=+37.205851793 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls") pod "dns-default-p4szr" (UID: "f621c1ae-a9a4-4c7d-979c-adfbcafaf33a") : secret "dns-default-metrics-tls" not found
Sep 08 15:09:47.834567 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:47.834536    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18237a47_8e2c_42f5_b960_f56188420981.slice/crio-8d953e1a8b75162a6ae9adb91dfe246bf0c4fc7a6701598f559c37323279544c WatchSource:0}: Error finding container 8d953e1a8b75162a6ae9adb91dfe246bf0c4fc7a6701598f559c37323279544c: Status 404 returned error can't find the container with id 8d953e1a8b75162a6ae9adb91dfe246bf0c4fc7a6701598f559c37323279544c
Sep 08 15:09:48.101688 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:48.101610    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dlpvm"]
Sep 08 15:09:48.170304 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:09:48.170273    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ccbb705_396e_4f0f_8756_b1a81ae1a840.slice/crio-cc10309009bce04b596a0446c89b45596160ff732aa98b453eae3a007bb2f59f WatchSource:0}: Error finding container cc10309009bce04b596a0446c89b45596160ff732aa98b453eae3a007bb2f59f: Status 404 returned error can't find the container with id cc10309009bce04b596a0446c89b45596160ff732aa98b453eae3a007bb2f59f
Sep 08 15:09:48.569565 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:48.569529    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lbq79" event={"ID":"18237a47-8e2c-42f5-b960-f56188420981","Type":"ContainerStarted","Data":"8d953e1a8b75162a6ae9adb91dfe246bf0c4fc7a6701598f559c37323279544c"}
Sep 08 15:09:48.572846 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:48.572820    2575 generic.go:356] "Generic (PLEG): container finished" podID="e9b25500-9302-461f-9458-f2e4a5dfbcd7" containerID="f4b322e5542bffab51aa2a298ebf4bef7bdd9f04bce09b5c082ab5ae864c552a" exitCode=0
Sep 08 15:09:48.572957 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:48.572900    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fchvk" event={"ID":"e9b25500-9302-461f-9458-f2e4a5dfbcd7","Type":"ContainerDied","Data":"f4b322e5542bffab51aa2a298ebf4bef7bdd9f04bce09b5c082ab5ae864c552a"}
Sep 08 15:09:48.574207 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:48.574173    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dlpvm" event={"ID":"7ccbb705-396e-4f0f-8756-b1a81ae1a840","Type":"ContainerStarted","Data":"cc10309009bce04b596a0446c89b45596160ff732aa98b453eae3a007bb2f59f"}
Sep 08 15:09:49.136513 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:49.136453    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:49.137121 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:49.136602    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:09:49.137121 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:49.136663    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls podName:f621c1ae-a9a4-4c7d-979c-adfbcafaf33a nodeName:}" failed. No retries permitted until 2026-09-08 15:09:53.136648407 +0000 UTC m=+41.222598428 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls") pod "dns-default-p4szr" (UID: "f621c1ae-a9a4-4c7d-979c-adfbcafaf33a") : secret "dns-default-metrics-tls" not found
Sep 08 15:09:49.584547 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:49.584511    2575 generic.go:356] "Generic (PLEG): container finished" podID="e9b25500-9302-461f-9458-f2e4a5dfbcd7" containerID="7943329a21bdbe456e1a7d4191711c358562c6d6556ae9e6b0d4ac403f709be7" exitCode=0
Sep 08 15:09:49.584699 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:49.584559    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fchvk" event={"ID":"e9b25500-9302-461f-9458-f2e4a5dfbcd7","Type":"ContainerDied","Data":"7943329a21bdbe456e1a7d4191711c358562c6d6556ae9e6b0d4ac403f709be7"}
Sep 08 15:09:50.591310 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:50.591009    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fchvk" event={"ID":"e9b25500-9302-461f-9458-f2e4a5dfbcd7","Type":"ContainerStarted","Data":"da510b76f4b15c7e1e4ce490bbe57658b5c77e09f406b9f350eebb102b4ab297"}
Sep 08 15:09:50.592651 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:50.592624    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nvc47" event={"ID":"a2e96079-8d75-4cd8-b77a-49f86c67a787","Type":"ContainerStarted","Data":"2d4fb54ecfb7c43ffb3dc5aabb21f6cc30d95d6996e6fbe87543886b051b6872"}
Sep 08 15:09:50.667395 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:50.667340    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fchvk" podStartSLOduration=7.917433842 podStartE2EDuration="38.667324261s" totalImagesPullingTime="30.749890419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:09:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:50.632881044 +0000 UTC m=+38.718831097" watchObservedRunningTime="2026-09-08 15:09:50.667324261 +0000 UTC m=+38.753274303"
Sep 08 15:09:50.812340 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:50.812310    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:50.926824 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:50.926791    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:51.185939 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:51.185844    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:52.599392 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:52.599363    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dlpvm" event={"ID":"7ccbb705-396e-4f0f-8756-b1a81ae1a840","Type":"ContainerStarted","Data":"ee6bb85be83b25e072adf1f40403c2cf14c891e8d0f05a160426c9da625c5bbb"}
Sep 08 15:09:52.599778 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:52.599460    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:09:52.617587 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:52.617540    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nvc47" podStartSLOduration=3.265628622 podStartE2EDuration="7.617521744s" totalImagesPullingTime="4.351893122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:50.674096367 +0000 UTC m=+38.760046408" watchObservedRunningTime="2026-09-08 15:09:52.617521744 +0000 UTC m=+40.703471787"
Sep 08 15:09:52.617710 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:52.617688    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dlpvm" podStartSLOduration=36.378649692 podStartE2EDuration="40.617681613s" totalImagesPullingTime="4.239031921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:52.616619005 +0000 UTC m=+40.702569047" watchObservedRunningTime="2026-09-08 15:09:52.617681613 +0000 UTC m=+40.703631653"
Sep 08 15:09:53.164431 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:53.164396    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:09:53.164623 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:53.164552    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:09:53.164623 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:09:53.164620    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls podName:f621c1ae-a9a4-4c7d-979c-adfbcafaf33a nodeName:}" failed. No retries permitted until 2026-09-08 15:10:01.16460137 +0000 UTC m=+49.250551403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls") pod "dns-default-p4szr" (UID: "f621c1ae-a9a4-4c7d-979c-adfbcafaf33a") : secret "dns-default-metrics-tls" not found
Sep 08 15:09:55.608367 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:55.608329    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lbq79" event={"ID":"18237a47-8e2c-42f5-b960-f56188420981","Type":"ContainerStarted","Data":"2fa799b49fe6a20087f7be09be6862656a5e8649fb4de413f449759bb761b4c4"}
Sep 08 15:09:55.628034 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:55.627948    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lbq79" podStartSLOduration=36.096587509 podStartE2EDuration="43.627930272s" totalImagesPullingTime="7.531342763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:09:55.627723953 +0000 UTC m=+43.713673996" watchObservedRunningTime="2026-09-08 15:09:55.627930272 +0000 UTC m=+43.713880314"
Sep 08 15:09:59.684977 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:59.684940    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:09:59.786124 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:09:59.786097    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:10:01.216737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:10:01.216698    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:10:01.217110 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:10:01.216819    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:10:01.217110 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:10:01.216873    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls podName:f621c1ae-a9a4-4c7d-979c-adfbcafaf33a nodeName:}" failed. No retries permitted until 2026-09-08 15:10:17.216859509 +0000 UTC m=+65.302809543 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls") pod "dns-default-p4szr" (UID: "f621c1ae-a9a4-4c7d-979c-adfbcafaf33a") : secret "dns-default-metrics-tls" not found
Sep 08 15:10:01.333649 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:10:01.333619    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:10:01.428545 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:10:01.428519    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:10:10.549854 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:10:10.549823    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9w9cc"
Sep 08 15:10:10.704477 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:10:10.704447    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:10:10.806990 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:10:10.806925    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:10:10.893762 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:10:10.893740    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:10:14.928927 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:10:14.928897    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:10:17.218145 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:10:17.218106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:10:17.218613 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:10:17.218216    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:10:17.218613 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:10:17.218281    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls podName:f621c1ae-a9a4-4c7d-979c-adfbcafaf33a nodeName:}" failed. No retries permitted until 2026-09-08 15:10:49.21826679 +0000 UTC m=+97.304216810 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls") pod "dns-default-p4szr" (UID: "f621c1ae-a9a4-4c7d-979c-adfbcafaf33a") : secret "dns-default-metrics-tls" not found
Sep 08 15:10:18.023221 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:10:18.023180    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:10:18.023399 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:10:18.023308    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 08 15:10:18.023399 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:10:18.023390    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs podName:4f348729-57f1-40ab-ae97-85bfefa7d80e nodeName:}" failed. No retries permitted until 2026-09-08 15:11:22.023370491 +0000 UTC m=+130.109320524 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs") pod "network-metrics-daemon-9vwnd" (UID: "4f348729-57f1-40ab-ae97-85bfefa7d80e") : secret "metrics-daemon-secret" not found
Sep 08 15:10:23.604765 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:10:23.604733    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dlpvm"
Sep 08 15:10:49.315272 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:10:49.315188    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:10:49.315671 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:10:49.315326    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:10:49.315671 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:10:49.315394    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls podName:f621c1ae-a9a4-4c7d-979c-adfbcafaf33a nodeName:}" failed. No retries permitted until 2026-09-08 15:11:53.315376498 +0000 UTC m=+161.401326539 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls") pod "dns-default-p4szr" (UID: "f621c1ae-a9a4-4c7d-979c-adfbcafaf33a") : secret "dns-default-metrics-tls" not found
Sep 08 15:11:22.026001 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:11:22.025960    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:11:22.026573 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:11:22.026108    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 08 15:11:22.026573 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:11:22.026175    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs podName:4f348729-57f1-40ab-ae97-85bfefa7d80e nodeName:}" failed. No retries permitted until 2026-09-08 15:13:24.026156126 +0000 UTC m=+252.112106167 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs") pod "network-metrics-daemon-9vwnd" (UID: "4f348729-57f1-40ab-ae97-85bfefa7d80e") : secret "metrics-daemon-secret" not found
Sep 08 15:11:48.356356 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:11:48.356311    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-p4szr" podUID="f621c1ae-a9a4-4c7d-979c-adfbcafaf33a"
Sep 08 15:11:48.442898 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:11:48.442863    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9vwnd" podUID="4f348729-57f1-40ab-ae97-85bfefa7d80e"
Sep 08 15:11:53.331884 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:11:53.331822    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:11:53.332379 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:11:53.331961    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:11:53.332379 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:11:53.332038    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls podName:f621c1ae-a9a4-4c7d-979c-adfbcafaf33a nodeName:}" failed. No retries permitted until 2026-09-08 15:13:55.332024794 +0000 UTC m=+283.417974814 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls") pod "dns-default-p4szr" (UID: "f621c1ae-a9a4-4c7d-979c-adfbcafaf33a") : secret "dns-default-metrics-tls" not found
Sep 08 15:12:02.427400 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:02.427354    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:12:02.427895 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:02.427531    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p4szr"
Sep 08 15:12:32.672865 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:32.672828    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:12:51.502625 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.502589    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"]
Sep 08 15:12:51.504240 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.504224    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"
Sep 08 15:12:51.507457 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.507437    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:12:51.513467 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.513449    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 08 15:12:51.514444 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.514426    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 08 15:12:51.514562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.514512    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-4tchr\""
Sep 08 15:12:51.515086 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.515072    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 08 15:12:51.536518 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.536469    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"]
Sep 08 15:12:51.547027 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.547005    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-nljdd"]
Sep 08 15:12:51.548771 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.548754    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"]
Sep 08 15:12:51.548885 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.548866    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.550410 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.550393    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:12:51.551433 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.551418    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-n4h5k\""
Sep 08 15:12:51.551949 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.551937    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 08 15:12:51.552408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.552390    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 08 15:12:51.553046 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.553025    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 08 15:12:51.554705 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.554686    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 08 15:12:51.554806 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.554716    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-knqhs\""
Sep 08 15:12:51.554806 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.554754    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 08 15:12:51.554806 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.554753    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 08 15:12:51.554983 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.554699    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 08 15:12:51.556393 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.556376    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:12:51.560477 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.560458    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 08 15:12:51.563086 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.563067    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-nljdd"]
Sep 08 15:12:51.569088 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.569069    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"]
Sep 08 15:12:51.596595 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.596562    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-skc22"]
Sep 08 15:12:51.598399 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.598379    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-8g8pd"]
Sep 08 15:12:51.598535 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.598517    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-skc22"
Sep 08 15:12:51.600059 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.600042    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.602189 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.602170    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 08 15:12:51.602596 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.602579    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8pvmv\""
Sep 08 15:12:51.602876 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.602861    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:12:51.602949 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.602934    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-drkr9\""
Sep 08 15:12:51.603093 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.603056    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 08 15:12:51.603093 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.603065    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 08 15:12:51.603328 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.603316    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 08 15:12:51.603964 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.603936    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 08 15:12:51.615775 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.615758    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 08 15:12:51.620528 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.620507    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nrgz\" (UniqueName: \"kubernetes.io/projected/742c48dd-0b32-418e-afb4-61f32954fbfd-kube-api-access-9nrgz\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cwfvk\" (UID: \"742c48dd-0b32-418e-afb4-61f32954fbfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"
Sep 08 15:12:51.620586 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.620545    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/742c48dd-0b32-418e-afb4-61f32954fbfd-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cwfvk\" (UID: \"742c48dd-0b32-418e-afb4-61f32954fbfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"
Sep 08 15:12:51.620636 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.620614    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/742c48dd-0b32-418e-afb4-61f32954fbfd-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cwfvk\" (UID: \"742c48dd-0b32-418e-afb4-61f32954fbfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"
Sep 08 15:12:51.624223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.624199    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-skc22"]
Sep 08 15:12:51.627416 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.627394    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-8g8pd"]
Sep 08 15:12:51.682520 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.682472    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-xnr24"]
Sep 08 15:12:51.684330 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.684312    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:12:51.687012 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.686982    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 08 15:12:51.687012 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.687006    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 08 15:12:51.687219 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.687036    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8lq8x\""
Sep 08 15:12:51.696015 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.695988    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-xnr24"]
Sep 08 15:12:51.721698 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.721665    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h78ll\" (UniqueName: \"kubernetes.io/projected/003e44e4-48a6-4f23-8469-a4345edf59a5-kube-api-access-h78ll\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:12:51.721698 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.721695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/65032153-eabb-470d-ac9c-0490d906994f-snapshots\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.721908 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.721716    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65032153-eabb-470d-ac9c-0490d906994f-tmp\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.721908 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.721732    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/65032153-eabb-470d-ac9c-0490d906994f-serving-cert\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.721908 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.721801    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nrgz\" (UniqueName: \"kubernetes.io/projected/742c48dd-0b32-418e-afb4-61f32954fbfd-kube-api-access-9nrgz\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cwfvk\" (UID: \"742c48dd-0b32-418e-afb4-61f32954fbfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"
Sep 08 15:12:51.721908 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.721854    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/742c48dd-0b32-418e-afb4-61f32954fbfd-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cwfvk\" (UID: \"742c48dd-0b32-418e-afb4-61f32954fbfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"
Sep 08 15:12:51.721908 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.721881    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b8rv\" (UniqueName: \"kubernetes.io/projected/3bf2739d-5bf1-43ee-a4a2-e2c75041d761-kube-api-access-7b8rv\") pod \"console-operator-85b454874b-nljdd\" (UID: \"3bf2739d-5bf1-43ee-a4a2-e2c75041d761\") " pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.722093 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.721910    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/003e44e4-48a6-4f23-8469-a4345edf59a5-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:12:51.722093 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.721935    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3bf2739d-5bf1-43ee-a4a2-e2c75041d761-config\") pod \"console-operator-85b454874b-nljdd\" (UID: \"3bf2739d-5bf1-43ee-a4a2-e2c75041d761\") " pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.722093 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.721951    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3bf2739d-5bf1-43ee-a4a2-e2c75041d761-serving-cert\") pod \"console-operator-85b454874b-nljdd\" (UID: \"3bf2739d-5bf1-43ee-a4a2-e2c75041d761\") " pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.722093 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.721971    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3bf2739d-5bf1-43ee-a4a2-e2c75041d761-trusted-ca\") pod \"console-operator-85b454874b-nljdd\" (UID: \"3bf2739d-5bf1-43ee-a4a2-e2c75041d761\") " pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.722093 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.722058    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:12:51.722093 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.722085    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65032153-eabb-470d-ac9c-0490d906994f-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.722302 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.722102    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85r9d\" (UniqueName: \"kubernetes.io/projected/65032153-eabb-470d-ac9c-0490d906994f-kube-api-access-85r9d\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.722302 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.722132    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/742c48dd-0b32-418e-afb4-61f32954fbfd-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cwfvk\" (UID: \"742c48dd-0b32-418e-afb4-61f32954fbfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"
Sep 08 15:12:51.722302 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.722153    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65032153-eabb-470d-ac9c-0490d906994f-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.722302 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.722172    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bldq4\" (UniqueName: \"kubernetes.io/projected/3e322dfd-60a8-4a4a-90c3-7ac16052fac2-kube-api-access-bldq4\") pod \"volume-data-source-validator-695c895bd6-skc22\" (UID: \"3e322dfd-60a8-4a4a-90c3-7ac16052fac2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-skc22"
Sep 08 15:12:51.722482 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.722365    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/742c48dd-0b32-418e-afb4-61f32954fbfd-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cwfvk\" (UID: \"742c48dd-0b32-418e-afb4-61f32954fbfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"
Sep 08 15:12:51.725034 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.725016    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/742c48dd-0b32-418e-afb4-61f32954fbfd-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cwfvk\" (UID: \"742c48dd-0b32-418e-afb4-61f32954fbfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"
Sep 08 15:12:51.734622 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.734599    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nrgz\" (UniqueName: \"kubernetes.io/projected/742c48dd-0b32-418e-afb4-61f32954fbfd-kube-api-access-9nrgz\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cwfvk\" (UID: \"742c48dd-0b32-418e-afb4-61f32954fbfd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"
Sep 08 15:12:51.780286 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.780189    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:12:51.813067 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.813038    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"
Sep 08 15:12:51.823363 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823338    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bldq4\" (UniqueName: \"kubernetes.io/projected/3e322dfd-60a8-4a4a-90c3-7ac16052fac2-kube-api-access-bldq4\") pod \"volume-data-source-validator-695c895bd6-skc22\" (UID: \"3e322dfd-60a8-4a4a-90c3-7ac16052fac2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-skc22"
Sep 08 15:12:51.823454 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823378    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-xnr24\" (UID: \"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:12:51.823533 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823474    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h78ll\" (UniqueName: \"kubernetes.io/projected/003e44e4-48a6-4f23-8469-a4345edf59a5-kube-api-access-h78ll\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:12:51.823589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823527    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/65032153-eabb-470d-ac9c-0490d906994f-snapshots\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.823589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823559    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65032153-eabb-470d-ac9c-0490d906994f-tmp\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.823589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823580    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/65032153-eabb-470d-ac9c-0490d906994f-serving-cert\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.823737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823639    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b8rv\" (UniqueName: \"kubernetes.io/projected/3bf2739d-5bf1-43ee-a4a2-e2c75041d761-kube-api-access-7b8rv\") pod \"console-operator-85b454874b-nljdd\" (UID: \"3bf2739d-5bf1-43ee-a4a2-e2c75041d761\") " pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.823737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823673    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/003e44e4-48a6-4f23-8469-a4345edf59a5-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:12:51.823737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823697    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-nginx-conf\") pod \"networking-console-plugin-79977f9844-xnr24\" (UID: \"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:12:51.823737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823735    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3bf2739d-5bf1-43ee-a4a2-e2c75041d761-config\") pod \"console-operator-85b454874b-nljdd\" (UID: \"3bf2739d-5bf1-43ee-a4a2-e2c75041d761\") " pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.823940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823758    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3bf2739d-5bf1-43ee-a4a2-e2c75041d761-serving-cert\") pod \"console-operator-85b454874b-nljdd\" (UID: \"3bf2739d-5bf1-43ee-a4a2-e2c75041d761\") " pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.823940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823786    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3bf2739d-5bf1-43ee-a4a2-e2c75041d761-trusted-ca\") pod \"console-operator-85b454874b-nljdd\" (UID: \"3bf2739d-5bf1-43ee-a4a2-e2c75041d761\") " pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.823940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823826    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:12:51.823940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823848    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65032153-eabb-470d-ac9c-0490d906994f-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.823940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823871    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85r9d\" (UniqueName: \"kubernetes.io/projected/65032153-eabb-470d-ac9c-0490d906994f-kube-api-access-85r9d\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.823940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.823911    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65032153-eabb-470d-ac9c-0490d906994f-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.824234 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.824169    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65032153-eabb-470d-ac9c-0490d906994f-tmp\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.824293 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.824240    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/65032153-eabb-470d-ac9c-0490d906994f-snapshots\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.824293 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:51.824274    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:12:51.824391 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:51.824344    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls podName:003e44e4-48a6-4f23-8469-a4345edf59a5 nodeName:}" failed. No retries permitted until 2026-09-08 15:12:52.324324394 +0000 UTC m=+220.410274415 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lqb82" (UID: "003e44e4-48a6-4f23-8469-a4345edf59a5") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:12:51.824572 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.824547    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65032153-eabb-470d-ac9c-0490d906994f-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.824833 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.824808    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/003e44e4-48a6-4f23-8469-a4345edf59a5-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:12:51.825333 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.825309    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65032153-eabb-470d-ac9c-0490d906994f-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.825421 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.825351    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3bf2739d-5bf1-43ee-a4a2-e2c75041d761-config\") pod \"console-operator-85b454874b-nljdd\" (UID: \"3bf2739d-5bf1-43ee-a4a2-e2c75041d761\") " pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.825588 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.825570    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3bf2739d-5bf1-43ee-a4a2-e2c75041d761-trusted-ca\") pod \"console-operator-85b454874b-nljdd\" (UID: \"3bf2739d-5bf1-43ee-a4a2-e2c75041d761\") " pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.826725 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.826697    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3bf2739d-5bf1-43ee-a4a2-e2c75041d761-serving-cert\") pod \"console-operator-85b454874b-nljdd\" (UID: \"3bf2739d-5bf1-43ee-a4a2-e2c75041d761\") " pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.826817 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.826800    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/65032153-eabb-470d-ac9c-0490d906994f-serving-cert\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.835679 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.835649    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h78ll\" (UniqueName: \"kubernetes.io/projected/003e44e4-48a6-4f23-8469-a4345edf59a5-kube-api-access-h78ll\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:12:51.835876 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.835860    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85r9d\" (UniqueName: \"kubernetes.io/projected/65032153-eabb-470d-ac9c-0490d906994f-kube-api-access-85r9d\") pod \"insights-operator-76bcd6ccf5-8g8pd\" (UID: \"65032153-eabb-470d-ac9c-0490d906994f\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.835929 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.835912    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b8rv\" (UniqueName: \"kubernetes.io/projected/3bf2739d-5bf1-43ee-a4a2-e2c75041d761-kube-api-access-7b8rv\") pod \"console-operator-85b454874b-nljdd\" (UID: \"3bf2739d-5bf1-43ee-a4a2-e2c75041d761\") " pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.835975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.835956    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bldq4\" (UniqueName: \"kubernetes.io/projected/3e322dfd-60a8-4a4a-90c3-7ac16052fac2-kube-api-access-bldq4\") pod \"volume-data-source-validator-695c895bd6-skc22\" (UID: \"3e322dfd-60a8-4a4a-90c3-7ac16052fac2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-skc22"
Sep 08 15:12:51.858886 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.858855    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:12:51.881866 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.881779    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:12:51.908257 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.908216    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-skc22"
Sep 08 15:12:51.912748 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.912723    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd"
Sep 08 15:12:51.924820 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.924697    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-xnr24\" (UID: \"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:12:51.924919 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.924855    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-nginx-conf\") pod \"networking-console-plugin-79977f9844-xnr24\" (UID: \"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:12:51.925045 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:51.925029    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 15:12:51.926064 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:51.925434    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert podName:b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3 nodeName:}" failed. No retries permitted until 2026-09-08 15:12:52.425404505 +0000 UTC m=+220.511354541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-xnr24" (UID: "b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3") : secret "networking-console-plugin-cert" not found
Sep 08 15:12:51.926985 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:51.926625    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-nginx-conf\") pod \"networking-console-plugin-79977f9844-xnr24\" (UID: \"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:12:52.084909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:52.082324    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk"]
Sep 08 15:12:52.087591 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:12:52.087563    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod742c48dd_0b32_418e_afb4_61f32954fbfd.slice/crio-de2bd4e0fa524f6cdcb5424c3653a915087d9c0381bd93e9c994cc984622ad16 WatchSource:0}: Error finding container de2bd4e0fa524f6cdcb5424c3653a915087d9c0381bd93e9c994cc984622ad16: Status 404 returned error can't find the container with id de2bd4e0fa524f6cdcb5424c3653a915087d9c0381bd93e9c994cc984622ad16
Sep 08 15:12:52.129477 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:52.129448    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-nljdd"]
Sep 08 15:12:52.132667 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:12:52.132639    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf2739d_5bf1_43ee_a4a2_e2c75041d761.slice/crio-a2cb240f93497582b24b9962e1517bfdb34ce69d4824b8d8420e0d54ce8ac4ce WatchSource:0}: Error finding container a2cb240f93497582b24b9962e1517bfdb34ce69d4824b8d8420e0d54ce8ac4ce: Status 404 returned error can't find the container with id a2cb240f93497582b24b9962e1517bfdb34ce69d4824b8d8420e0d54ce8ac4ce
Sep 08 15:12:52.181474 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:52.181447    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-skc22"]
Sep 08 15:12:52.184638 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:52.184611    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-8g8pd"]
Sep 08 15:12:52.185128 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:12:52.185102    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e322dfd_60a8_4a4a_90c3_7ac16052fac2.slice/crio-1848a8532a6f8e240aaf122e9f09d78a523b0d53140e80c72bc0d32512a58b37 WatchSource:0}: Error finding container 1848a8532a6f8e240aaf122e9f09d78a523b0d53140e80c72bc0d32512a58b37: Status 404 returned error can't find the container with id 1848a8532a6f8e240aaf122e9f09d78a523b0d53140e80c72bc0d32512a58b37
Sep 08 15:12:52.188864 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:12:52.188848    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65032153_eabb_470d_ac9c_0490d906994f.slice/crio-3a5c04a1d752d202a3a04e2071108225f0e51b57ee05b6a41b8070ad222cfd4e WatchSource:0}: Error finding container 3a5c04a1d752d202a3a04e2071108225f0e51b57ee05b6a41b8070ad222cfd4e: Status 404 returned error can't find the container with id 3a5c04a1d752d202a3a04e2071108225f0e51b57ee05b6a41b8070ad222cfd4e
Sep 08 15:12:52.328364 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:52.328320    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:12:52.328553 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:52.328435    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:12:52.328553 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:52.328516    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls podName:003e44e4-48a6-4f23-8469-a4345edf59a5 nodeName:}" failed. No retries permitted until 2026-09-08 15:12:53.328479417 +0000 UTC m=+221.414429438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lqb82" (UID: "003e44e4-48a6-4f23-8469-a4345edf59a5") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:12:52.428715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:52.428689    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-xnr24\" (UID: \"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:12:52.428875 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:52.428819    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 15:12:52.428875 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:52.428874    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert podName:b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3 nodeName:}" failed. No retries permitted until 2026-09-08 15:12:53.4288612 +0000 UTC m=+221.514811233 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-xnr24" (UID: "b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3") : secret "networking-console-plugin-cert" not found
Sep 08 15:12:53.025313 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:53.025272    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-skc22" event={"ID":"3e322dfd-60a8-4a4a-90c3-7ac16052fac2","Type":"ContainerStarted","Data":"1848a8532a6f8e240aaf122e9f09d78a523b0d53140e80c72bc0d32512a58b37"}
Sep 08 15:12:53.027112 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:53.027070    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk" event={"ID":"742c48dd-0b32-418e-afb4-61f32954fbfd","Type":"ContainerStarted","Data":"de2bd4e0fa524f6cdcb5424c3653a915087d9c0381bd93e9c994cc984622ad16"}
Sep 08 15:12:53.028805 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:53.028689    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd" event={"ID":"65032153-eabb-470d-ac9c-0490d906994f","Type":"ContainerStarted","Data":"3a5c04a1d752d202a3a04e2071108225f0e51b57ee05b6a41b8070ad222cfd4e"}
Sep 08 15:12:53.031360 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:53.031335    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nljdd" event={"ID":"3bf2739d-5bf1-43ee-a4a2-e2c75041d761","Type":"ContainerStarted","Data":"a2cb240f93497582b24b9962e1517bfdb34ce69d4824b8d8420e0d54ce8ac4ce"}
Sep 08 15:12:53.336201 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:53.336084    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:12:53.336371 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:53.336322    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:12:53.336436 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:53.336415    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls podName:003e44e4-48a6-4f23-8469-a4345edf59a5 nodeName:}" failed. No retries permitted until 2026-09-08 15:12:55.336394882 +0000 UTC m=+223.422344922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lqb82" (UID: "003e44e4-48a6-4f23-8469-a4345edf59a5") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:12:53.437402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:53.437369    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-xnr24\" (UID: \"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:12:53.437615 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:53.437597    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 15:12:53.437699 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:53.437667    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert podName:b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3 nodeName:}" failed. No retries permitted until 2026-09-08 15:12:55.437647961 +0000 UTC m=+223.523597984 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-xnr24" (UID: "b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3") : secret "networking-console-plugin-cert" not found
Sep 08 15:12:54.035853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:54.035813    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-skc22" event={"ID":"3e322dfd-60a8-4a4a-90c3-7ac16052fac2","Type":"ContainerStarted","Data":"1fcbf9e7fd510b1e2ec6769690ec3f1bbb6a656ed617324d6c00a5a857595bfb"}
Sep 08 15:12:54.053155 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:54.053068    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-skc22" podStartSLOduration=1.834112508 podStartE2EDuration="3.053052068s" totalImagesPullingTime="1.21893956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:12:54.052080123 +0000 UTC m=+222.138030166" watchObservedRunningTime="2026-09-08 15:12:54.053052068 +0000 UTC m=+222.139002140"
Sep 08 15:12:54.133762 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:54.133722    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:12:54.196688 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:54.196649    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:12:55.351812 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:55.351700    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:12:55.352202 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:55.351879    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:12:55.352202 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:55.351953    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls podName:003e44e4-48a6-4f23-8469-a4345edf59a5 nodeName:}" failed. No retries permitted until 2026-09-08 15:12:59.351932199 +0000 UTC m=+227.437882236 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lqb82" (UID: "003e44e4-48a6-4f23-8469-a4345edf59a5") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:12:55.452740 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:55.452688    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-xnr24\" (UID: \"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:12:55.452913 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:55.452885    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 15:12:55.453061 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:55.452962    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert podName:b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3 nodeName:}" failed. No retries permitted until 2026-09-08 15:12:59.452942293 +0000 UTC m=+227.538892333 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-xnr24" (UID: "b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3") : secret "networking-console-plugin-cert" not found
Sep 08 15:12:56.043670 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.043629    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd" event={"ID":"65032153-eabb-470d-ac9c-0490d906994f","Type":"ContainerStarted","Data":"b3d0040efa54e8782f0570c58552df560848580ae58a5d0b13d0c4a84d113584"}
Sep 08 15:12:56.045296 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.045277    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/0.log"
Sep 08 15:12:56.045427 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.045310    2575 generic.go:356] "Generic (PLEG): container finished" podID="3bf2739d-5bf1-43ee-a4a2-e2c75041d761" containerID="c1e04257a806d3edf0aa8e408a08fcc39cc19b7640575faa6a37b48f9d21a9eb" exitCode=255
Sep 08 15:12:56.045427 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.045396    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nljdd" event={"ID":"3bf2739d-5bf1-43ee-a4a2-e2c75041d761","Type":"ContainerDied","Data":"c1e04257a806d3edf0aa8e408a08fcc39cc19b7640575faa6a37b48f9d21a9eb"}
Sep 08 15:12:56.045592 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.045577    2575 scope.go:117] "RemoveContainer" containerID="c1e04257a806d3edf0aa8e408a08fcc39cc19b7640575faa6a37b48f9d21a9eb"
Sep 08 15:12:56.047029 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.047010    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk" event={"ID":"742c48dd-0b32-418e-afb4-61f32954fbfd","Type":"ContainerStarted","Data":"d8ec0d0c62bdc943bf14dc8592492b7715ccc93959fcc28c232d9d5f11f2d543"}
Sep 08 15:12:56.063012 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.062961    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd" podStartSLOduration=1.913326219 podStartE2EDuration="5.062943671s" totalImagesPullingTime="3.149617452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:12:56.062204699 +0000 UTC m=+224.148154742" watchObservedRunningTime="2026-09-08 15:12:56.062943671 +0000 UTC m=+224.148893712"
Sep 08 15:12:56.084740 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.084690    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk" podStartSLOduration=1.83502492 podStartE2EDuration="5.084669688s" totalImagesPullingTime="3.249644768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:12:56.084300771 +0000 UTC m=+224.170250811" watchObservedRunningTime="2026-09-08 15:12:56.084669688 +0000 UTC m=+224.170619731"
Sep 08 15:12:56.667697 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.667659    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5wrcn"]
Sep 08 15:12:56.669780 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.669758    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:56.675046 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.675015    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qrxjt\""
Sep 08 15:12:56.675157 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.675109    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 08 15:12:56.676875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.676856    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 08 15:12:56.690913 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.690882    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5wrcn"]
Sep 08 15:12:56.762346 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.762313    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:56.762346 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.762351    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:56.762572 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.762408    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24d5bfe3-9e61-4007-8057-377731d84ad3-data-volume\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:56.762572 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.762436    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24d5bfe3-9e61-4007-8057-377731d84ad3-crio-socket\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:56.762572 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.762473    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv25t\" (UniqueName: \"kubernetes.io/projected/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-api-access-nv25t\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:56.863365 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.863310    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24d5bfe3-9e61-4007-8057-377731d84ad3-data-volume\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:56.863513 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.863391    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24d5bfe3-9e61-4007-8057-377731d84ad3-crio-socket\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:56.863513 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.863461    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv25t\" (UniqueName: \"kubernetes.io/projected/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-api-access-nv25t\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:56.863596 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.863534    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:56.863596 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.863538    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24d5bfe3-9e61-4007-8057-377731d84ad3-crio-socket\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:56.863596 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.863562    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:56.863732 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:56.863656    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:12:56.863732 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:56.863692    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:12:56.863732 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:56.863715    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls podName:24d5bfe3-9e61-4007-8057-377731d84ad3 nodeName:}" failed. No retries permitted until 2026-09-08 15:12:57.363695364 +0000 UTC m=+225.449645384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5wrcn" (UID: "24d5bfe3-9e61-4007-8057-377731d84ad3") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:12:56.863861 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.863739    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24d5bfe3-9e61-4007-8057-377731d84ad3-data-volume\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:56.863861 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:56.863775    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm podName:24d5bfe3-9e61-4007-8057-377731d84ad3 nodeName:}" failed. No retries permitted until 2026-09-08 15:12:57.363756221 +0000 UTC m=+225.449706264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5wrcn" (UID: "24d5bfe3-9e61-4007-8057-377731d84ad3") : configmap "kube-rbac-proxy" not found
Sep 08 15:12:56.872301 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:56.872269    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv25t\" (UniqueName: \"kubernetes.io/projected/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-api-access-nv25t\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:57.051984 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.051910    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/1.log"
Sep 08 15:12:57.052358 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.052340    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/0.log"
Sep 08 15:12:57.052428 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.052381    2575 generic.go:356] "Generic (PLEG): container finished" podID="3bf2739d-5bf1-43ee-a4a2-e2c75041d761" containerID="0b4abca6de9d89481194954655ffaf39972dcd8945daf0f22c0e61acd08f7a2a" exitCode=255
Sep 08 15:12:57.052479 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.052417    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nljdd" event={"ID":"3bf2739d-5bf1-43ee-a4a2-e2c75041d761","Type":"ContainerDied","Data":"0b4abca6de9d89481194954655ffaf39972dcd8945daf0f22c0e61acd08f7a2a"}
Sep 08 15:12:57.052543 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.052502    2575 scope.go:117] "RemoveContainer" containerID="c1e04257a806d3edf0aa8e408a08fcc39cc19b7640575faa6a37b48f9d21a9eb"
Sep 08 15:12:57.052777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.052757    2575 scope.go:117] "RemoveContainer" containerID="0b4abca6de9d89481194954655ffaf39972dcd8945daf0f22c0e61acd08f7a2a"
Sep 08 15:12:57.052958 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:57.052939    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-nljdd_openshift-console-operator(3bf2739d-5bf1-43ee-a4a2-e2c75041d761)\"" pod="openshift-console-operator/console-operator-85b454874b-nljdd" podUID="3bf2739d-5bf1-43ee-a4a2-e2c75041d761"
Sep 08 15:12:57.368210 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.368178    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:57.368349 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.368215    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:57.368349 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:57.368304    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:12:57.368429 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:57.368371    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm podName:24d5bfe3-9e61-4007-8057-377731d84ad3 nodeName:}" failed. No retries permitted until 2026-09-08 15:12:58.368355777 +0000 UTC m=+226.454305801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5wrcn" (UID: "24d5bfe3-9e61-4007-8057-377731d84ad3") : configmap "kube-rbac-proxy" not found
Sep 08 15:12:57.368429 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:57.368393    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:12:57.368521 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:57.368454    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls podName:24d5bfe3-9e61-4007-8057-377731d84ad3 nodeName:}" failed. No retries permitted until 2026-09-08 15:12:58.368441083 +0000 UTC m=+226.454391131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5wrcn" (UID: "24d5bfe3-9e61-4007-8057-377731d84ad3") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:12:57.564672 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.564638    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-89stb"]
Sep 08 15:12:57.567368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.567349    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-89stb"
Sep 08 15:12:57.570697 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.570676    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 08 15:12:57.570877 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.570859    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-7gzf9\""
Sep 08 15:12:57.571176 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.571162    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 08 15:12:57.589632 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.589603    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-89stb"]
Sep 08 15:12:57.670250 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.670155    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8jpd\" (UniqueName: \"kubernetes.io/projected/2d8c8142-9b59-4ac4-8f4b-96d48e3f40cd-kube-api-access-x8jpd\") pod \"migrator-fdbb8fdff-89stb\" (UID: \"2d8c8142-9b59-4ac4-8f4b-96d48e3f40cd\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-89stb"
Sep 08 15:12:57.771032 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.770985    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8jpd\" (UniqueName: \"kubernetes.io/projected/2d8c8142-9b59-4ac4-8f4b-96d48e3f40cd-kube-api-access-x8jpd\") pod \"migrator-fdbb8fdff-89stb\" (UID: \"2d8c8142-9b59-4ac4-8f4b-96d48e3f40cd\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-89stb"
Sep 08 15:12:57.779958 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.779930    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8jpd\" (UniqueName: \"kubernetes.io/projected/2d8c8142-9b59-4ac4-8f4b-96d48e3f40cd-kube-api-access-x8jpd\") pod \"migrator-fdbb8fdff-89stb\" (UID: \"2d8c8142-9b59-4ac4-8f4b-96d48e3f40cd\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-89stb"
Sep 08 15:12:57.876507 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:57.876440    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-89stb"
Sep 08 15:12:58.056747 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:58.056719    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/1.log"
Sep 08 15:12:58.057111 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:58.057090    2575 scope.go:117] "RemoveContainer" containerID="0b4abca6de9d89481194954655ffaf39972dcd8945daf0f22c0e61acd08f7a2a"
Sep 08 15:12:58.057283 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:58.057265    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-nljdd_openshift-console-operator(3bf2739d-5bf1-43ee-a4a2-e2c75041d761)\"" pod="openshift-console-operator/console-operator-85b454874b-nljdd" podUID="3bf2739d-5bf1-43ee-a4a2-e2c75041d761"
Sep 08 15:12:58.141994 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:58.141635    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-89stb"]
Sep 08 15:12:58.144354 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:12:58.144327    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d8c8142_9b59_4ac4_8f4b_96d48e3f40cd.slice/crio-8b2363aae62e7cf5d9fb483fa0e0ab796ec83226d9ecbbcf578a2e7b4211bc64 WatchSource:0}: Error finding container 8b2363aae62e7cf5d9fb483fa0e0ab796ec83226d9ecbbcf578a2e7b4211bc64: Status 404 returned error can't find the container with id 8b2363aae62e7cf5d9fb483fa0e0ab796ec83226d9ecbbcf578a2e7b4211bc64
Sep 08 15:12:58.375930 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:58.375900    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:58.376090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:58.375936    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:12:58.376090 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:58.376027    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:12:58.376090 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:58.376047    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:12:58.376292 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:58.376098    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm podName:24d5bfe3-9e61-4007-8057-377731d84ad3 nodeName:}" failed. No retries permitted until 2026-09-08 15:13:00.37608176 +0000 UTC m=+228.462031780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5wrcn" (UID: "24d5bfe3-9e61-4007-8057-377731d84ad3") : configmap "kube-rbac-proxy" not found
Sep 08 15:12:58.376292 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:58.376115    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls podName:24d5bfe3-9e61-4007-8057-377731d84ad3 nodeName:}" failed. No retries permitted until 2026-09-08 15:13:00.376108517 +0000 UTC m=+228.462058577 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5wrcn" (UID: "24d5bfe3-9e61-4007-8057-377731d84ad3") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:12:58.409684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:58.409658    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4ltq9_d33f854f-48e2-45ca-8289-0f5cb0817be7/dns-node-resolver/0.log"
Sep 08 15:12:59.060873 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:59.060840    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-89stb" event={"ID":"2d8c8142-9b59-4ac4-8f4b-96d48e3f40cd","Type":"ContainerStarted","Data":"8b2363aae62e7cf5d9fb483fa0e0ab796ec83226d9ecbbcf578a2e7b4211bc64"}
Sep 08 15:12:59.128953 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:59.128912    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:12:59.198975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:59.198930    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:12:59.384534 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:59.384480    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:12:59.384697 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:59.384595    2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:12:59.384697 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:59.384667    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls podName:003e44e4-48a6-4f23-8469-a4345edf59a5 nodeName:}" failed. No retries permitted until 2026-09-08 15:13:07.384651569 +0000 UTC m=+235.470601588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-lqb82" (UID: "003e44e4-48a6-4f23-8469-a4345edf59a5") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:12:59.485581 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:59.485535    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-xnr24\" (UID: \"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:12:59.485757 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:59.485697    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 15:12:59.485819 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:12:59.485767    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert podName:b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3 nodeName:}" failed. No retries permitted until 2026-09-08 15:13:07.485752349 +0000 UTC m=+235.571702368 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-xnr24" (UID: "b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3") : secret "networking-console-plugin-cert" not found
Sep 08 15:12:59.609466 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:12:59.609441    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qvxxd_dea9369e-6c20-4b59-a04b-eb20d2f8cca5/node-ca/0.log"
Sep 08 15:13:00.065591 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:00.065485    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-89stb" event={"ID":"2d8c8142-9b59-4ac4-8f4b-96d48e3f40cd","Type":"ContainerStarted","Data":"550a6b31b3da405dea2647f511172827584eaa307c984e41915160ca67c9cf86"}
Sep 08 15:13:00.065591 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:00.065570    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-89stb" event={"ID":"2d8c8142-9b59-4ac4-8f4b-96d48e3f40cd","Type":"ContainerStarted","Data":"2fa150f571585175f51b3ebdaf5b05cedd018386772b0e048cb3450a673b5e5c"}
Sep 08 15:13:00.085844 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:00.085791    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-89stb" podStartSLOduration=1.488844743 podStartE2EDuration="3.085776299s" totalImagesPullingTime="1.596931556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:12:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:13:00.085270219 +0000 UTC m=+228.171220271" watchObservedRunningTime="2026-09-08 15:13:00.085776299 +0000 UTC m=+228.171726341"
Sep 08 15:13:00.393038 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:00.393005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:13:00.393199 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:00.393045    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:13:00.393199 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:00.393168    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:13:00.393199 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:00.393189    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:13:00.393344 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:00.393225    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls podName:24d5bfe3-9e61-4007-8057-377731d84ad3 nodeName:}" failed. No retries permitted until 2026-09-08 15:13:04.393209197 +0000 UTC m=+232.479159222 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5wrcn" (UID: "24d5bfe3-9e61-4007-8057-377731d84ad3") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:13:00.393344 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:00.393257    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm podName:24d5bfe3-9e61-4007-8057-377731d84ad3 nodeName:}" failed. No retries permitted until 2026-09-08 15:13:04.393238407 +0000 UTC m=+232.479188444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5wrcn" (UID: "24d5bfe3-9e61-4007-8057-377731d84ad3") : configmap "kube-rbac-proxy" not found
Sep 08 15:13:00.409278 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:00.409249    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nvc47_a2e96079-8d75-4cd8-b77a-49f86c67a787/serve-healthcheck-canary/0.log"
Sep 08 15:13:01.859782 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:01.859749    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:13:01.860151 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:01.859806    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:13:01.860151 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:01.860145    2575 scope.go:117] "RemoveContainer" containerID="0b4abca6de9d89481194954655ffaf39972dcd8945daf0f22c0e61acd08f7a2a"
Sep 08 15:13:01.860302 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:01.860284    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-nljdd_openshift-console-operator(3bf2739d-5bf1-43ee-a4a2-e2c75041d761)\"" pod="openshift-console-operator/console-operator-85b454874b-nljdd" podUID="3bf2739d-5bf1-43ee-a4a2-e2c75041d761"
Sep 08 15:13:02.429934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:02.429897    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:02.506836 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:02.506805    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:02.649435 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:02.649404    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:02.839119 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:02.839033    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:03.003010 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:03.002976    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:03.251946 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:03.251904    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:03.841975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:03.841943    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:04.152542 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:04.152506    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:04.254711 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:04.254675    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:04.427260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:04.427163    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:13:04.427260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:04.427208    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:13:04.427479 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:04.427281    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:13:04.427479 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:04.427297    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:13:04.427479 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:04.427350    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm podName:24d5bfe3-9e61-4007-8057-377731d84ad3 nodeName:}" failed. No retries permitted until 2026-09-08 15:13:12.427335135 +0000 UTC m=+240.513285155 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5wrcn" (UID: "24d5bfe3-9e61-4007-8057-377731d84ad3") : configmap "kube-rbac-proxy" not found
Sep 08 15:13:04.427479 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:04.427428    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls podName:24d5bfe3-9e61-4007-8057-377731d84ad3 nodeName:}" failed. No retries permitted until 2026-09-08 15:13:12.427403829 +0000 UTC m=+240.513353864 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5wrcn" (UID: "24d5bfe3-9e61-4007-8057-377731d84ad3") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:13:04.445715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:04.445685    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:04.635503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:04.635463    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:05.170264 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:05.170223    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:05.271465 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:05.271430    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:05.638037 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:05.638002    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:06.238855 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:06.238819    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:06.473611 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:06.473579    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:06.632357 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:06.632328    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:06.833532 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:06.833468    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:06.901387 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:06.901295    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:07.445956 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:07.445919    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:07.453000 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:07.452972    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:13:07.455277 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:07.455255    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/003e44e4-48a6-4f23-8469-a4345edf59a5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-lqb82\" (UID: \"003e44e4-48a6-4f23-8469-a4345edf59a5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:13:07.464090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:07.464069    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"
Sep 08 15:13:07.554332 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:07.554299    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-xnr24\" (UID: \"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:13:07.554472 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:07.554450    2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 15:13:07.554538 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:07.554527    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert podName:b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3 nodeName:}" failed. No retries permitted until 2026-09-08 15:13:23.554510853 +0000 UTC m=+251.640460894 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-xnr24" (UID: "b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3") : secret "networking-console-plugin-cert" not found
Sep 08 15:13:07.732757 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:07.732681    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82"]
Sep 08 15:13:07.735914 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:13:07.735890    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod003e44e4_48a6_4f23_8469_a4345edf59a5.slice/crio-319516f7c7ebb9bd60ff5945c86766280ce5bbdb58b5ae7072e13ca0a72cb252 WatchSource:0}: Error finding container 319516f7c7ebb9bd60ff5945c86766280ce5bbdb58b5ae7072e13ca0a72cb252: Status 404 returned error can't find the container with id 319516f7c7ebb9bd60ff5945c86766280ce5bbdb58b5ae7072e13ca0a72cb252
Sep 08 15:13:07.836872 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:07.836832    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:08.028806 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:08.028719    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:08.095720 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:08.095684    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82" event={"ID":"003e44e4-48a6-4f23-8469-a4345edf59a5","Type":"ContainerStarted","Data":"319516f7c7ebb9bd60ff5945c86766280ce5bbdb58b5ae7072e13ca0a72cb252"}
Sep 08 15:13:08.449420 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:08.449385    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:09.036368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:09.036320    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:09.631753 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:09.631709    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:09.751671 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:09.751637    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:09.853350 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:09.853321    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:09.970602 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:09.970519    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:10.071337 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:10.071255    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:10.103203 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:10.103168    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82" event={"ID":"003e44e4-48a6-4f23-8469-a4345edf59a5","Type":"ContainerStarted","Data":"21ea54a44b753bbba9be952737c693e4ec0391c19e3405aaf1b8065efd7d7b10"}
Sep 08 15:13:10.120518 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:10.120449    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-lqb82" podStartSLOduration=17.210680902 podStartE2EDuration="19.120433665s" totalImagesPullingTime="1.909752763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:13:10.119792882 +0000 UTC m=+238.205742926" watchObservedRunningTime="2026-09-08 15:13:10.120433665 +0000 UTC m=+238.206383706"
Sep 08 15:13:10.139000 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:10.138961    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:10.433737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:10.433700    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:12.493543 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:12.493515    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:13:12.493946 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:12.493551    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:13:12.494133 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:12.494111    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24d5bfe3-9e61-4007-8057-377731d84ad3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:13:12.496000 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:12.495978    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24d5bfe3-9e61-4007-8057-377731d84ad3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5wrcn\" (UID: \"24d5bfe3-9e61-4007-8057-377731d84ad3\") " pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:13:12.581366 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:12.581336    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qrxjt\""
Sep 08 15:13:12.588746 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:12.588729    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5wrcn"
Sep 08 15:13:12.861422 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:12.861220    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5wrcn"]
Sep 08 15:13:12.864039 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:13:12.864011    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d5bfe3_9e61_4007_8057_377731d84ad3.slice/crio-61449cb6a35a1e1bce68fb66103d3075d624be6e9a26a86af43049fc3df12988 WatchSource:0}: Error finding container 61449cb6a35a1e1bce68fb66103d3075d624be6e9a26a86af43049fc3df12988: Status 404 returned error can't find the container with id 61449cb6a35a1e1bce68fb66103d3075d624be6e9a26a86af43049fc3df12988
Sep 08 15:13:13.113700 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:13.113616    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5wrcn" event={"ID":"24d5bfe3-9e61-4007-8057-377731d84ad3","Type":"ContainerStarted","Data":"ad10edc6a3a37d0ff79acddbdcede7c747ed317f1172e510fc38069b5881724a"}
Sep 08 15:13:13.113700 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:13.113676    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5wrcn" event={"ID":"24d5bfe3-9e61-4007-8057-377731d84ad3","Type":"ContainerStarted","Data":"61449cb6a35a1e1bce68fb66103d3075d624be6e9a26a86af43049fc3df12988"}
Sep 08 15:13:13.423254 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:13.423157    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:13.491048 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:13.491011    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:15.121020 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:15.120974    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5wrcn" event={"ID":"24d5bfe3-9e61-4007-8057-377731d84ad3","Type":"ContainerStarted","Data":"44fde512bcffb2d2425a4f94b19f7079f566330f05f8f38d8a9ff130acf57556"}
Sep 08 15:13:16.126787 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:16.126746    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5wrcn" event={"ID":"24d5bfe3-9e61-4007-8057-377731d84ad3","Type":"ContainerStarted","Data":"62cec519dfcc0b3701c5abfee53bc83ecec4c5f67fa279c963920e731c5e228c"}
Sep 08 15:13:16.146102 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:16.146042    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5wrcn" podStartSLOduration=17.216140434 podStartE2EDuration="20.14602474s" totalImagesPullingTime="2.929884306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:12:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:13:16.144865699 +0000 UTC m=+244.230815742" watchObservedRunningTime="2026-09-08 15:13:16.14602474 +0000 UTC m=+244.231974828"
Sep 08 15:13:17.425716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:17.425678    2575 scope.go:117] "RemoveContainer" containerID="0b4abca6de9d89481194954655ffaf39972dcd8945daf0f22c0e61acd08f7a2a"
Sep 08 15:13:18.134547 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:18.134510    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:13:18.134948 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:18.134930    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/1.log"
Sep 08 15:13:18.135007 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:18.134967    2575 generic.go:356] "Generic (PLEG): container finished" podID="3bf2739d-5bf1-43ee-a4a2-e2c75041d761" containerID="15cf152da1cfb3fa741375982966ff8e6b3080dd0bf591f2ca8ee484d3882ff2" exitCode=255
Sep 08 15:13:18.135078 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:18.135051    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nljdd" event={"ID":"3bf2739d-5bf1-43ee-a4a2-e2c75041d761","Type":"ContainerDied","Data":"15cf152da1cfb3fa741375982966ff8e6b3080dd0bf591f2ca8ee484d3882ff2"}
Sep 08 15:13:18.135143 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:18.135131    2575 scope.go:117] "RemoveContainer" containerID="0b4abca6de9d89481194954655ffaf39972dcd8945daf0f22c0e61acd08f7a2a"
Sep 08 15:13:18.135534 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:18.135516    2575 scope.go:117] "RemoveContainer" containerID="15cf152da1cfb3fa741375982966ff8e6b3080dd0bf591f2ca8ee484d3882ff2"
Sep 08 15:13:18.135699 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:18.135677    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-nljdd_openshift-console-operator(3bf2739d-5bf1-43ee-a4a2-e2c75041d761)\"" pod="openshift-console-operator/console-operator-85b454874b-nljdd" podUID="3bf2739d-5bf1-43ee-a4a2-e2c75041d761"
Sep 08 15:13:19.143543 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:19.143513    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:13:20.560000 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.559967    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c4d6bc854-bq6mc"]
Sep 08 15:13:20.563093 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.563072    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.565894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.565860    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 08 15:13:20.566578 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.566560    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2m6vp\""
Sep 08 15:13:20.566867 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.566851    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 08 15:13:20.568131 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.568113    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 08 15:13:20.572152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.572130    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 08 15:13:20.585599 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.585571    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c4d6bc854-bq6mc"]
Sep 08 15:13:20.596066 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.596027    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:20.656104 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.656061    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-ca-trust-extracted\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.656104 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.656106    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-installation-pull-secrets\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.656364 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.656128    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-trusted-ca\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.656364 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.656277    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-bound-sa-token\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.656364 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.656318    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf8v5\" (UniqueName: \"kubernetes.io/projected/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-kube-api-access-hf8v5\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.656524 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.656398    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-image-registry-private-configuration\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.656524 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.656424    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-registry-tls\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.656524 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.656448    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-registry-certificates\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.689465 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.689416    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:20.757400 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.757359    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-bound-sa-token\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.757400 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.757401    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf8v5\" (UniqueName: \"kubernetes.io/projected/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-kube-api-access-hf8v5\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.757684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.757436    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-image-registry-private-configuration\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.757684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.757461    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-registry-tls\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.757684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.757483    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-registry-certificates\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.757684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.757540    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-ca-trust-extracted\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.757684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.757577    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-installation-pull-secrets\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.757684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.757601    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-trusted-ca\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.758079 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.758040    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-ca-trust-extracted\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.758446 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.758425    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-registry-certificates\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.758759 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.758736    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-trusted-ca\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.760019 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.759986    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-installation-pull-secrets\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.760264 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.760246    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-image-registry-private-configuration\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.760390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.760374    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-registry-tls\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.765441 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.765414    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-bound-sa-token\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.765564 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.765537    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf8v5\" (UniqueName: \"kubernetes.io/projected/17a6542b-38ff-4c74-aa24-cf8f5f3845a3-kube-api-access-hf8v5\") pod \"image-registry-6c4d6bc854-bq6mc\" (UID: \"17a6542b-38ff-4c74-aa24-cf8f5f3845a3\") " pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:20.872154 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:20.872121    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:21.148669 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:21.148598    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c4d6bc854-bq6mc"]
Sep 08 15:13:21.152110 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:13:21.152084    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17a6542b_38ff_4c74_aa24_cf8f5f3845a3.slice/crio-6a00e531580cc3bd678cb22860909b007e4970b88fa33b66d13fd48e944e25da WatchSource:0}: Error finding container 6a00e531580cc3bd678cb22860909b007e4970b88fa33b66d13fd48e944e25da: Status 404 returned error can't find the container with id 6a00e531580cc3bd678cb22860909b007e4970b88fa33b66d13fd48e944e25da
Sep 08 15:13:21.859504 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:21.859466    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:13:21.859889 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:21.859535    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:13:21.859889 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:21.859852    2575 scope.go:117] "RemoveContainer" containerID="15cf152da1cfb3fa741375982966ff8e6b3080dd0bf591f2ca8ee484d3882ff2"
Sep 08 15:13:21.860019 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:21.860001    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-nljdd_openshift-console-operator(3bf2739d-5bf1-43ee-a4a2-e2c75041d761)\"" pod="openshift-console-operator/console-operator-85b454874b-nljdd" podUID="3bf2739d-5bf1-43ee-a4a2-e2c75041d761"
Sep 08 15:13:22.155469 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:22.155388    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc" event={"ID":"17a6542b-38ff-4c74-aa24-cf8f5f3845a3","Type":"ContainerStarted","Data":"868e3ce24cd5027b496a5ce4cce2be43e201d3f1986c734aa8dbc32be384eb37"}
Sep 08 15:13:22.155469 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:22.155434    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc" event={"ID":"17a6542b-38ff-4c74-aa24-cf8f5f3845a3","Type":"ContainerStarted","Data":"6a00e531580cc3bd678cb22860909b007e4970b88fa33b66d13fd48e944e25da"}
Sep 08 15:13:22.155681 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:22.155506    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:13:22.177550 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:22.177479    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc" podStartSLOduration=2.17746182 podStartE2EDuration="2.17746182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:13:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:13:22.176450862 +0000 UTC m=+250.262400908" watchObservedRunningTime="2026-09-08 15:13:22.17746182 +0000 UTC m=+250.263411861"
Sep 08 15:13:23.153146 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:23.153069    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:23.229270 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:23.229231    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:23.384964 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:23.384923    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:23.578777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:23.578681    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-xnr24\" (UID: \"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:13:23.581117 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:23.581082    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-xnr24\" (UID: \"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:13:23.795385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:23.795347    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8lq8x\""
Sep 08 15:13:23.803641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:23.803615    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24"
Sep 08 15:13:24.074374 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:24.074340    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-xnr24"]
Sep 08 15:13:24.077426 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:13:24.077395    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2eb0e27_7888_4a1c_bbc8_e7aed028d5a3.slice/crio-1fca556612ce8870a4d82d433be44cba327d51462f580944ff84a67a98725612 WatchSource:0}: Error finding container 1fca556612ce8870a4d82d433be44cba327d51462f580944ff84a67a98725612: Status 404 returned error can't find the container with id 1fca556612ce8870a4d82d433be44cba327d51462f580944ff84a67a98725612
Sep 08 15:13:24.084227 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:24.084198    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:13:24.086614 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:24.086590    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f348729-57f1-40ab-ae97-85bfefa7d80e-metrics-certs\") pod \"network-metrics-daemon-9vwnd\" (UID: \"4f348729-57f1-40ab-ae97-85bfefa7d80e\") " pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:13:24.163087 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:24.163050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24" event={"ID":"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3","Type":"ContainerStarted","Data":"1fca556612ce8870a4d82d433be44cba327d51462f580944ff84a67a98725612"}
Sep 08 15:13:24.330804 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:24.330766    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t2q2k\""
Sep 08 15:13:24.339012 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:24.338984    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vwnd"
Sep 08 15:13:24.607970 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:24.607936    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9vwnd"]
Sep 08 15:13:24.610998 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:13:24.610972    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f348729_57f1_40ab_ae97_85bfefa7d80e.slice/crio-220ff40ffbcf62edddea41b4348ebf97a407f6d0ca1ab8d84e266a5c6abf397f WatchSource:0}: Error finding container 220ff40ffbcf62edddea41b4348ebf97a407f6d0ca1ab8d84e266a5c6abf397f: Status 404 returned error can't find the container with id 220ff40ffbcf62edddea41b4348ebf97a407f6d0ca1ab8d84e266a5c6abf397f
Sep 08 15:13:25.167056 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:25.167023    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9vwnd" event={"ID":"4f348729-57f1-40ab-ae97-85bfefa7d80e","Type":"ContainerStarted","Data":"220ff40ffbcf62edddea41b4348ebf97a407f6d0ca1ab8d84e266a5c6abf397f"}
Sep 08 15:13:25.735203 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:25.735155    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:26.171797 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:26.171762    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9vwnd" event={"ID":"4f348729-57f1-40ab-ae97-85bfefa7d80e","Type":"ContainerStarted","Data":"9c272cd5ded1a96aa6a8f19049eb0af088b81b9b9ed1a6868e69f14e600051e0"}
Sep 08 15:13:26.172178 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:26.171814    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9vwnd" event={"ID":"4f348729-57f1-40ab-ae97-85bfefa7d80e","Type":"ContainerStarted","Data":"b1fe7700a2f1cb277c0bfced668495c5c954bfe03b6813acadf72e28590db878"}
Sep 08 15:13:26.196597 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:26.196540    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9vwnd" podStartSLOduration=253.037371874 podStartE2EDuration="4m14.196522672s" totalImagesPullingTime="1.159150798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:13:26.195473336 +0000 UTC m=+254.281423392" watchObservedRunningTime="2026-09-08 15:13:26.196522672 +0000 UTC m=+254.282472707"
Sep 08 15:13:26.300296 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:26.300257    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:26.400620 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:26.400585    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:27.545292 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.545257    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"]
Sep 08 15:13:27.549663 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.549621    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.553705 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.553303    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 08 15:13:27.553705 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.553387    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-wzl79\""
Sep 08 15:13:27.553705 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.553303    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 08 15:13:27.554485 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.554451    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 08 15:13:27.556868 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.556844    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"]
Sep 08 15:13:27.560753 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.560731    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bvj8f"]
Sep 08 15:13:27.560902 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.560883    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.564167 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.564145    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7v8cq\""
Sep 08 15:13:27.564780 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.564579    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 08 15:13:27.564780 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.564686    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"]
Sep 08 15:13:27.564780 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.564684    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 08 15:13:27.564780 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.564779    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.565027 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.564817    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 08 15:13:27.567543 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.567524    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 08 15:13:27.567660 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.567556    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 08 15:13:27.567660 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.567539    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 08 15:13:27.568019 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.567556    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5v7mq\""
Sep 08 15:13:27.580749 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.580723    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"]
Sep 08 15:13:27.616023 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.615989    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0f2c8a0-259e-4e85-b78f-42be03476971-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-9s9nz\" (UID: \"d0f2c8a0-259e-4e85-b78f-42be03476971\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.616207 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.616038    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbldw\" (UniqueName: \"kubernetes.io/projected/d0f2c8a0-259e-4e85-b78f-42be03476971-kube-api-access-pbldw\") pod \"openshift-state-metrics-657fd6c54-9s9nz\" (UID: \"d0f2c8a0-259e-4e85-b78f-42be03476971\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.616207 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.616059    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/531d0205-8276-4123-be0b-cd0056cd2aef-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.616207 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.616145    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/531d0205-8276-4123-be0b-cd0056cd2aef-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.616326 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.616210    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/531d0205-8276-4123-be0b-cd0056cd2aef-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.616326 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.616249    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0f2c8a0-259e-4e85-b78f-42be03476971-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-9s9nz\" (UID: \"d0f2c8a0-259e-4e85-b78f-42be03476971\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.616326 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.616282    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5fp9\" (UniqueName: \"kubernetes.io/projected/531d0205-8276-4123-be0b-cd0056cd2aef-kube-api-access-r5fp9\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.616326 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.616317    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0f2c8a0-259e-4e85-b78f-42be03476971-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-9s9nz\" (UID: \"d0f2c8a0-259e-4e85-b78f-42be03476971\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.616509 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.616345    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/531d0205-8276-4123-be0b-cd0056cd2aef-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.616509 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.616413    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/531d0205-8276-4123-be0b-cd0056cd2aef-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.717299 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717262    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-accelerators-collector-config\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.717468 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717309    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0f2c8a0-259e-4e85-b78f-42be03476971-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-9s9nz\" (UID: \"d0f2c8a0-259e-4e85-b78f-42be03476971\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.717468 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717379    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-root\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.717468 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717449    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-wtmp\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.717648 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717476    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-sys\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.717648 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717513    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbldw\" (UniqueName: \"kubernetes.io/projected/d0f2c8a0-259e-4e85-b78f-42be03476971-kube-api-access-pbldw\") pod \"openshift-state-metrics-657fd6c54-9s9nz\" (UID: \"d0f2c8a0-259e-4e85-b78f-42be03476971\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.717648 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717577    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/531d0205-8276-4123-be0b-cd0056cd2aef-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.717789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717666    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/531d0205-8276-4123-be0b-cd0056cd2aef-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.717789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717696    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbdhv\" (UniqueName: \"kubernetes.io/projected/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-kube-api-access-zbdhv\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.717789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717722    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/531d0205-8276-4123-be0b-cd0056cd2aef-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.717789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717747    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0f2c8a0-259e-4e85-b78f-42be03476971-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-9s9nz\" (UID: \"d0f2c8a0-259e-4e85-b78f-42be03476971\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.717789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717773    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-metrics-client-ca\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.718019 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717801    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5fp9\" (UniqueName: \"kubernetes.io/projected/531d0205-8276-4123-be0b-cd0056cd2aef-kube-api-access-r5fp9\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.718019 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717836    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0f2c8a0-259e-4e85-b78f-42be03476971-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-9s9nz\" (UID: \"d0f2c8a0-259e-4e85-b78f-42be03476971\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.718019 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.717973    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/531d0205-8276-4123-be0b-cd0056cd2aef-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.718159 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.718076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0f2c8a0-259e-4e85-b78f-42be03476971-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-9s9nz\" (UID: \"d0f2c8a0-259e-4e85-b78f-42be03476971\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.718159 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.718126    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-tls\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.718260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.718162    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/531d0205-8276-4123-be0b-cd0056cd2aef-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.718260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.718196    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-textfile\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.718260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.718233    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/531d0205-8276-4123-be0b-cd0056cd2aef-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.718408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.718264    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.718466 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.718415    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/531d0205-8276-4123-be0b-cd0056cd2aef-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.718969 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.718914    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/531d0205-8276-4123-be0b-cd0056cd2aef-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.721170 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.721139    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0f2c8a0-259e-4e85-b78f-42be03476971-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-9s9nz\" (UID: \"d0f2c8a0-259e-4e85-b78f-42be03476971\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.721261 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.721179    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/531d0205-8276-4123-be0b-cd0056cd2aef-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.721261 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.721190    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/531d0205-8276-4123-be0b-cd0056cd2aef-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.721559 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.721544    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0f2c8a0-259e-4e85-b78f-42be03476971-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-9s9nz\" (UID: \"d0f2c8a0-259e-4e85-b78f-42be03476971\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.727738 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.727680    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbldw\" (UniqueName: \"kubernetes.io/projected/d0f2c8a0-259e-4e85-b78f-42be03476971-kube-api-access-pbldw\") pod \"openshift-state-metrics-657fd6c54-9s9nz\" (UID: \"d0f2c8a0-259e-4e85-b78f-42be03476971\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.727738 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.727711    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5fp9\" (UniqueName: \"kubernetes.io/projected/531d0205-8276-4123-be0b-cd0056cd2aef-kube-api-access-r5fp9\") pod \"kube-state-metrics-74cb57df7-xw6hd\" (UID: \"531d0205-8276-4123-be0b-cd0056cd2aef\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:27.819741 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.819650    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-accelerators-collector-config\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.819909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.819800    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-root\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.819909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.819836    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-wtmp\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.819909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.819868    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-sys\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.820068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.819917    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbdhv\" (UniqueName: \"kubernetes.io/projected/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-kube-api-access-zbdhv\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.820068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.819927    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-root\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.820068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.819946    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-metrics-client-ca\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.820068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.820001    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-sys\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.820068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.820020    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-wtmp\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.820068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.820005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-tls\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.820346 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:27.820086    2575 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 08 15:13:27.820346 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.820094    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-textfile\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.820346 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.820121    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.820346 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:27.820155    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-tls podName:98a5f71a-b85d-4ca6-ba83-2b8cb7835687 nodeName:}" failed. No retries permitted until 2026-09-08 15:13:28.320135256 +0000 UTC m=+256.406085290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-tls") pod "node-exporter-bvj8f" (UID: "98a5f71a-b85d-4ca6-ba83-2b8cb7835687") : secret "node-exporter-tls" not found
Sep 08 15:13:27.820580 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.820355    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-accelerators-collector-config\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.820580 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.820539    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-textfile\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.820580 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.820565    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-metrics-client-ca\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.822508 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.822470    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.828988 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.828953    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbdhv\" (UniqueName: \"kubernetes.io/projected/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-kube-api-access-zbdhv\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:27.863805 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.863765    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"
Sep 08 15:13:27.873709 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:27.873682    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"
Sep 08 15:13:28.148540 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:28.148474    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz"]
Sep 08 15:13:28.150897 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:13:28.150871    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0f2c8a0_259e_4e85_b78f_42be03476971.slice/crio-b76270370140328c7e7acc440dea888ca92aed70ee5140ac8342d0d231ce5e33 WatchSource:0}: Error finding container b76270370140328c7e7acc440dea888ca92aed70ee5140ac8342d0d231ce5e33: Status 404 returned error can't find the container with id b76270370140328c7e7acc440dea888ca92aed70ee5140ac8342d0d231ce5e33
Sep 08 15:13:28.154726 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:28.154700    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd"]
Sep 08 15:13:28.159184 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:13:28.159157    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod531d0205_8276_4123_be0b_cd0056cd2aef.slice/crio-5cc7a4f00907bd7425ce000648c85656edf0afaf8cbf6ffea78e0a59eb45b644 WatchSource:0}: Error finding container 5cc7a4f00907bd7425ce000648c85656edf0afaf8cbf6ffea78e0a59eb45b644: Status 404 returned error can't find the container with id 5cc7a4f00907bd7425ce000648c85656edf0afaf8cbf6ffea78e0a59eb45b644
Sep 08 15:13:28.179005 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:28.178971    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd" event={"ID":"531d0205-8276-4123-be0b-cd0056cd2aef","Type":"ContainerStarted","Data":"5cc7a4f00907bd7425ce000648c85656edf0afaf8cbf6ffea78e0a59eb45b644"}
Sep 08 15:13:28.180119 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:28.180089    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz" event={"ID":"d0f2c8a0-259e-4e85-b78f-42be03476971","Type":"ContainerStarted","Data":"b76270370140328c7e7acc440dea888ca92aed70ee5140ac8342d0d231ce5e33"}
Sep 08 15:13:28.181564 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:28.181538    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24" event={"ID":"b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3","Type":"ContainerStarted","Data":"57590cf545344c0fb41338704ff02ec5b3c77f597ed4c5bc8952f32e2b460d5c"}
Sep 08 15:13:28.198062 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:28.198016    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-xnr24" podStartSLOduration=33.348755178 podStartE2EDuration="37.198000116s" totalImagesPullingTime="3.849244938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:13:28.19702887 +0000 UTC m=+256.282978912" watchObservedRunningTime="2026-09-08 15:13:28.198000116 +0000 UTC m=+256.283950157"
Sep 08 15:13:28.324404 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:28.324362    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-tls\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:28.326781 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:28.326757    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/98a5f71a-b85d-4ca6-ba83-2b8cb7835687-node-exporter-tls\") pod \"node-exporter-bvj8f\" (UID: \"98a5f71a-b85d-4ca6-ba83-2b8cb7835687\") " pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:28.479073 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:28.478987    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bvj8f"
Sep 08 15:13:28.491148 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:13:28.491101    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a5f71a_b85d_4ca6_ba83_2b8cb7835687.slice/crio-6b8713f852bf89dec8a3b5900f38a45ad7b75766bdbe0eef404b8a15c690ef39 WatchSource:0}: Error finding container 6b8713f852bf89dec8a3b5900f38a45ad7b75766bdbe0eef404b8a15c690ef39: Status 404 returned error can't find the container with id 6b8713f852bf89dec8a3b5900f38a45ad7b75766bdbe0eef404b8a15c690ef39
Sep 08 15:13:29.188836 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.188792    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bvj8f" event={"ID":"98a5f71a-b85d-4ca6-ba83-2b8cb7835687","Type":"ContainerStarted","Data":"6b8713f852bf89dec8a3b5900f38a45ad7b75766bdbe0eef404b8a15c690ef39"}
Sep 08 15:13:29.192480 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.192371    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz" event={"ID":"d0f2c8a0-259e-4e85-b78f-42be03476971","Type":"ContainerStarted","Data":"c016a72541f5990926f103b12ed62e9683bbcce71fa5da20a82178903cc57bb3"}
Sep 08 15:13:29.192480 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.192443    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz" event={"ID":"d0f2c8a0-259e-4e85-b78f-42be03476971","Type":"ContainerStarted","Data":"2c5a65d466a33a875f196c7b4b440adf4d709d059f4e088bc2dbd58cf5988a68"}
Sep 08 15:13:29.644396 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.644358    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"]
Sep 08 15:13:29.648619 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.648585    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.651660 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.651386    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 08 15:13:29.651660 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.651421    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 08 15:13:29.651660 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.651467    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 08 15:13:29.651660 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.651564    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 08 15:13:29.651660 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.651583    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-cvng0tlcp6qlv\""
Sep 08 15:13:29.652015 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.651784    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ct9ks\""
Sep 08 15:13:29.652015 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.651810    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 08 15:13:29.657962 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.657937    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"]
Sep 08 15:13:29.740505 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.740447    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/191eb204-5730-4132-a3fc-4c3fa119f2c4-metrics-client-ca\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.740701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.740518    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.740701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.740551    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.740701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.740628    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.740856 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.740708    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-tls\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.740856 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.740735    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.740856 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.740755    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-grpc-tls\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.740856 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.740781    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmlcq\" (UniqueName: \"kubernetes.io/projected/191eb204-5730-4132-a3fc-4c3fa119f2c4-kube-api-access-bmlcq\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.818185 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.818143    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:29.842233 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.842183    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/191eb204-5730-4132-a3fc-4c3fa119f2c4-metrics-client-ca\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.842233 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.842225    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.842513 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.842249    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.842513 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.842408    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.842631 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.842514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-tls\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.842631 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.842556    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.842631 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.842586    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-grpc-tls\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.842631 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.842612    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmlcq\" (UniqueName: \"kubernetes.io/projected/191eb204-5730-4132-a3fc-4c3fa119f2c4-kube-api-access-bmlcq\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.843235 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.843077    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/191eb204-5730-4132-a3fc-4c3fa119f2c4-metrics-client-ca\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.845532 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.845482    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.845632 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.845534    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.845632 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.845549    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.845710 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.845679    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.845745 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.845711    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-thanos-querier-tls\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.845955 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.845937    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/191eb204-5730-4132-a3fc-4c3fa119f2c4-secret-grpc-tls\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.852540 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.852476    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmlcq\" (UniqueName: \"kubernetes.io/projected/191eb204-5730-4132-a3fc-4c3fa119f2c4-kube-api-access-bmlcq\") pod \"thanos-querier-5fcdbd7b65-dfmjt\" (UID: \"191eb204-5730-4132-a3fc-4c3fa119f2c4\") " pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:29.879627 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.879586    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:29.960622 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:29.960468    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:30.198644 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:30.198606    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bvj8f" event={"ID":"98a5f71a-b85d-4ca6-ba83-2b8cb7835687","Type":"ContainerStarted","Data":"ff59824a4ab26815efee81610b4b318aedc4da1641c899a8ce48fef686f8215d"}
Sep 08 15:13:30.201279 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:30.201249    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd" event={"ID":"531d0205-8276-4123-be0b-cd0056cd2aef","Type":"ContainerStarted","Data":"754a7af6c8d5a3ee903f7ae5acaf5669313633a99341fac149f0bb31af18b7b9"}
Sep 08 15:13:30.204136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:30.204014    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz" event={"ID":"d0f2c8a0-259e-4e85-b78f-42be03476971","Type":"ContainerStarted","Data":"193b6ebb116b9c32aca8668a1f050b78a7498f1a7b70c8e2a2f396464efd99c8"}
Sep 08 15:13:30.241731 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:30.241680    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-9s9nz" podStartSLOduration=1.429830287 podStartE2EDuration="3.241660049s" totalImagesPullingTime="1.811829762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:13:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:13:30.240011978 +0000 UTC m=+258.325962021" watchObservedRunningTime="2026-09-08 15:13:30.241660049 +0000 UTC m=+258.327610093"
Sep 08 15:13:30.376344 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:30.376308    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"]
Sep 08 15:13:30.379959 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:13:30.379924    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191eb204_5730_4132_a3fc_4c3fa119f2c4.slice/crio-18726986ee7484b2f65635a13bcb25dc356af4d8afc48cfaa2dd03c479dbb168 WatchSource:0}: Error finding container 18726986ee7484b2f65635a13bcb25dc356af4d8afc48cfaa2dd03c479dbb168: Status 404 returned error can't find the container with id 18726986ee7484b2f65635a13bcb25dc356af4d8afc48cfaa2dd03c479dbb168
Sep 08 15:13:30.523828 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:30.523741    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:30.623217 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:30.623171    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:30.730437 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:30.730403    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:30.934748 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:30.934710    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:31.210371 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.210279    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd" event={"ID":"531d0205-8276-4123-be0b-cd0056cd2aef","Type":"ContainerStarted","Data":"a139f450327f6cde14b9b4eb481c5fa5632bcbf33e8eee4ab8ffe58637a29e3f"}
Sep 08 15:13:31.210371 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.210342    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd" event={"ID":"531d0205-8276-4123-be0b-cd0056cd2aef","Type":"ContainerStarted","Data":"cb68b4d5d4081e1f3966bda6b94d918c842512e8498a32a57a8363433c4b3730"}
Sep 08 15:13:31.211619 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.211594    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt" event={"ID":"191eb204-5730-4132-a3fc-4c3fa119f2c4","Type":"ContainerStarted","Data":"18726986ee7484b2f65635a13bcb25dc356af4d8afc48cfaa2dd03c479dbb168"}
Sep 08 15:13:31.213071 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.213050    2575 generic.go:356] "Generic (PLEG): container finished" podID="98a5f71a-b85d-4ca6-ba83-2b8cb7835687" containerID="ff59824a4ab26815efee81610b4b318aedc4da1641c899a8ce48fef686f8215d" exitCode=0
Sep 08 15:13:31.213146 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.213103    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bvj8f" event={"ID":"98a5f71a-b85d-4ca6-ba83-2b8cb7835687","Type":"ContainerDied","Data":"ff59824a4ab26815efee81610b4b318aedc4da1641c899a8ce48fef686f8215d"}
Sep 08 15:13:31.230611 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.230563    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xw6hd" podStartSLOduration=2.298530044 podStartE2EDuration="4.230520462s" totalImagesPullingTime="1.931990418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:13:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:13:31.229256016 +0000 UTC m=+259.315206062" watchObservedRunningTime="2026-09-08 15:13:31.230520462 +0000 UTC m=+259.316470506"
Sep 08 15:13:31.856915 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.856882    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-565b9cbcb7-722mx"]
Sep 08 15:13:31.860161 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.860145    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:31.863696 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.863659    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-x8zj2\""
Sep 08 15:13:31.863696 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.863659    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 08 15:13:31.863871 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.863717    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 08 15:13:31.863871 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.863735    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 08 15:13:31.863871 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.863750    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 08 15:13:31.864169 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.864154    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7vkqfvcddfjp5\""
Sep 08 15:13:31.869731 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.869710    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-565b9cbcb7-722mx"]
Sep 08 15:13:31.963300 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.963262    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a5cd3d-c334-41a7-b578-1960e91c49bb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:31.963300 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.963307    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d6a5cd3d-c334-41a7-b578-1960e91c49bb-audit-log\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:31.963516 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.963383    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6a5cd3d-c334-41a7-b578-1960e91c49bb-client-ca-bundle\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:31.963516 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.963427    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d6a5cd3d-c334-41a7-b578-1960e91c49bb-secret-metrics-server-client-certs\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:31.963516 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.963448    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d6a5cd3d-c334-41a7-b578-1960e91c49bb-metrics-server-audit-profiles\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:31.963645 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.963515    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d6a5cd3d-c334-41a7-b578-1960e91c49bb-secret-metrics-server-tls\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:31.963645 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:31.963538    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl56n\" (UniqueName: \"kubernetes.io/projected/d6a5cd3d-c334-41a7-b578-1960e91c49bb-kube-api-access-nl56n\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.064672 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.064634    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d6a5cd3d-c334-41a7-b578-1960e91c49bb-secret-metrics-server-tls\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.064672 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.064669    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl56n\" (UniqueName: \"kubernetes.io/projected/d6a5cd3d-c334-41a7-b578-1960e91c49bb-kube-api-access-nl56n\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.064870 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.064722    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a5cd3d-c334-41a7-b578-1960e91c49bb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.064870 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.064762    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d6a5cd3d-c334-41a7-b578-1960e91c49bb-audit-log\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.064870 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.064811    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6a5cd3d-c334-41a7-b578-1960e91c49bb-client-ca-bundle\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.064870 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.064843    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d6a5cd3d-c334-41a7-b578-1960e91c49bb-secret-metrics-server-client-certs\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.065029 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.064870    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d6a5cd3d-c334-41a7-b578-1960e91c49bb-metrics-server-audit-profiles\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.065274 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.065247    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d6a5cd3d-c334-41a7-b578-1960e91c49bb-audit-log\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.065683 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.065658    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6a5cd3d-c334-41a7-b578-1960e91c49bb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.066036 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.066014    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d6a5cd3d-c334-41a7-b578-1960e91c49bb-metrics-server-audit-profiles\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.067320 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.067294    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d6a5cd3d-c334-41a7-b578-1960e91c49bb-secret-metrics-server-client-certs\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.067418 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.067401    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d6a5cd3d-c334-41a7-b578-1960e91c49bb-secret-metrics-server-tls\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.067563 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.067539    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6a5cd3d-c334-41a7-b578-1960e91c49bb-client-ca-bundle\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.075147 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.075124    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl56n\" (UniqueName: \"kubernetes.io/projected/d6a5cd3d-c334-41a7-b578-1960e91c49bb-kube-api-access-nl56n\") pod \"metrics-server-565b9cbcb7-722mx\" (UID: \"d6a5cd3d-c334-41a7-b578-1960e91c49bb\") " pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.170033 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.169683    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:32.225699 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.225647    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bvj8f" event={"ID":"98a5f71a-b85d-4ca6-ba83-2b8cb7835687","Type":"ContainerStarted","Data":"ec09485468419f70bab7e32111631c76bfecba6a4981f808b7c8b35df504bd26"}
Sep 08 15:13:32.226058 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.225755    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bvj8f" event={"ID":"98a5f71a-b85d-4ca6-ba83-2b8cb7835687","Type":"ContainerStarted","Data":"6b02adedff3109e78029b8b43c3bee06dba6d666a1263ae23806ad04a99b05d5"}
Sep 08 15:13:32.249533 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.249463    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bvj8f" podStartSLOduration=3.567859432 podStartE2EDuration="5.24944844s" totalImagesPullingTime="1.600621057s" totalInitContainerRuntime="80.967951ms" isStatefulPod=true podCreationTimestamp="2026-09-08 15:13:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:13:32.248153531 +0000 UTC m=+260.334103568" watchObservedRunningTime="2026-09-08 15:13:32.24944844 +0000 UTC m=+260.335398482"
Sep 08 15:13:32.329103 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.329067    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:32.452578 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.452477    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-565b9cbcb7-722mx"]
Sep 08 15:13:32.455694 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:13:32.455667    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a5cd3d_c334_41a7_b578_1960e91c49bb.slice/crio-dd0704444c69f0041ec0c969272495453c4a595edbf53099bf2ff7539c934341 WatchSource:0}: Error finding container dd0704444c69f0041ec0c969272495453c4a595edbf53099bf2ff7539c934341: Status 404 returned error can't find the container with id dd0704444c69f0041ec0c969272495453c4a595edbf53099bf2ff7539c934341
Sep 08 15:13:32.658983 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:32.658943    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:33.231630 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.231589    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx" event={"ID":"d6a5cd3d-c334-41a7-b578-1960e91c49bb","Type":"ContainerStarted","Data":"dd0704444c69f0041ec0c969272495453c4a595edbf53099bf2ff7539c934341"}
Sep 08 15:13:33.426231 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.426196    2575 scope.go:117] "RemoveContainer" containerID="15cf152da1cfb3fa741375982966ff8e6b3080dd0bf591f2ca8ee484d3882ff2"
Sep 08 15:13:33.426427 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:13:33.426407    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-nljdd_openshift-console-operator(3bf2739d-5bf1-43ee-a4a2-e2c75041d761)\"" pod="openshift-console-operator/console-operator-85b454874b-nljdd" podUID="3bf2739d-5bf1-43ee-a4a2-e2c75041d761"
Sep 08 15:13:33.768265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.768230    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:33.776910 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.776875    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:13:33.781397 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.781372    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.786019 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.785932    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d8nh4eg2s042j\""
Sep 08 15:13:33.786301 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.786276    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 08 15:13:33.786301 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.786292    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 08 15:13:33.786705 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.786686    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 08 15:13:33.786813 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.786686    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 08 15:13:33.787346 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.787326    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 08 15:13:33.787944 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.787523    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 08 15:13:33.787944 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.787538    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 08 15:13:33.787944 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.787664    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 08 15:13:33.787944 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.787685    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p9rz7\""
Sep 08 15:13:33.787944 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.787763    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 08 15:13:33.787944 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.787799    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 08 15:13:33.787944 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.787813    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 08 15:13:33.787944 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.787834    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 08 15:13:33.788432 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.788056    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 08 15:13:33.789529 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.789509    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 08 15:13:33.800853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.800830    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:13:33.881005 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.880969    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881173 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881021    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881173 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881048    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881173 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881076    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881173 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881114    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/523e70ee-482d-4851-999c-ff3d8b35f1b5-config-out\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881173 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881142    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881427 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881200    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881427 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881229    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881427 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881252    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxfw2\" (UniqueName: \"kubernetes.io/projected/523e70ee-482d-4851-999c-ff3d8b35f1b5-kube-api-access-pxfw2\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881427 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881290    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/523e70ee-482d-4851-999c-ff3d8b35f1b5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881427 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881320    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881427 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881368    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881427 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881405    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881734 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881435    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881734 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881467    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881734 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881518    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881734 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881543    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-web-config\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881734 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881560    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881734 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881585    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-config\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.881734 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.881610    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.982546 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.982502    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.982724 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.982552    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.982724 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.982659    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.982724 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.982699    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.982908 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.982740    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/523e70ee-482d-4851-999c-ff3d8b35f1b5-config-out\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983038 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983013    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983098 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983066    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983151 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983096    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983151 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983129    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxfw2\" (UniqueName: \"kubernetes.io/projected/523e70ee-482d-4851-999c-ff3d8b35f1b5-kube-api-access-pxfw2\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983173    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983183    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/523e70ee-482d-4851-999c-ff3d8b35f1b5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983243    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983273    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983297    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983328    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983380    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983634 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983388    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983634 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983417    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983634 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983441    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-web-config\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983634 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983470    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983634 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983540    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-config\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.983634 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.983571    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.984772 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.984742    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.985895 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.985870    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.986650 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.986567    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.986731 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.986671    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.986816 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.986793    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/523e70ee-482d-4851-999c-ff3d8b35f1b5-config-out\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.986873 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.986815    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.987053 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.986935    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.987155 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.987105    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.988286 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.987841    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.988286 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.988226    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.990536 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.989156    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.990536 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.989710    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.990536 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.989743    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-web-config\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.991323 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.990827    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/523e70ee-482d-4851-999c-ff3d8b35f1b5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.991412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.991331    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-config\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.993085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.993062    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.995390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.995369    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:33.996747 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:33.996726    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxfw2\" (UniqueName: \"kubernetes.io/projected/523e70ee-482d-4851-999c-ff3d8b35f1b5-kube-api-access-pxfw2\") pod \"prometheus-k8s-0\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:34.094439 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:34.094350    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:34.238055 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:34.238013    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx" event={"ID":"d6a5cd3d-c334-41a7-b578-1960e91c49bb","Type":"ContainerStarted","Data":"35691ec31fdb960cc6054bf31097827f92d6251421b5c7843cf466e03b485dd0"}
Sep 08 15:13:34.256641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:34.256584    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx" podStartSLOduration=1.589971257 podStartE2EDuration="3.256568401s" totalImagesPullingTime="1.666597144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:13:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:13:34.256013964 +0000 UTC m=+262.341964002" watchObservedRunningTime="2026-09-08 15:13:34.256568401 +0000 UTC m=+262.342518445"
Sep 08 15:13:34.405756 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:34.405715    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:13:34.408883 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:13:34.408851    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod523e70ee_482d_4851_999c_ff3d8b35f1b5.slice/crio-ca53689810fe906bcff7f0217bf6e180ddc9735993c447bb79d0c3348609cc26 WatchSource:0}: Error finding container ca53689810fe906bcff7f0217bf6e180ddc9735993c447bb79d0c3348609cc26: Status 404 returned error can't find the container with id ca53689810fe906bcff7f0217bf6e180ddc9735993c447bb79d0c3348609cc26
Sep 08 15:13:35.244566 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:35.244529    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerStarted","Data":"ca53689810fe906bcff7f0217bf6e180ddc9735993c447bb79d0c3348609cc26"}
Sep 08 15:13:36.250804 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:36.250767    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt" event={"ID":"191eb204-5730-4132-a3fc-4c3fa119f2c4","Type":"ContainerStarted","Data":"d585441c2f447147d7d52e18e4d8de720c1f910cf2349276d1d684c877cacf1b"}
Sep 08 15:13:36.251215 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:36.250837    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt" event={"ID":"191eb204-5730-4132-a3fc-4c3fa119f2c4","Type":"ContainerStarted","Data":"29712e6253102e94922b8e3722d96c9600723054f441c1da9fb4c0b4322b6793"}
Sep 08 15:13:36.251215 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:36.250877    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt" event={"ID":"191eb204-5730-4132-a3fc-4c3fa119f2c4","Type":"ContainerStarted","Data":"cb54adab072e5765e21b1348bf08bc0a73d59639660958a0343c984389d231a6"}
Sep 08 15:13:36.252220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:36.252196    2575 generic.go:356] "Generic (PLEG): container finished" podID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerID="ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06" exitCode=0
Sep 08 15:13:36.252301 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:36.252252    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerDied","Data":"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"}
Sep 08 15:13:37.259403 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:37.259373    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt" event={"ID":"191eb204-5730-4132-a3fc-4c3fa119f2c4","Type":"ContainerStarted","Data":"3ff4e5bf6dbb4678aa43f8dca25c8fb29c133ebddc35a2a70e5345065e2f6ef3"}
Sep 08 15:13:37.259714 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:37.259464    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt" event={"ID":"191eb204-5730-4132-a3fc-4c3fa119f2c4","Type":"ContainerStarted","Data":"91034e3371ca87677523f9e1235620c2ba9038756e3cb3caa48ca5fcf19809e9"}
Sep 08 15:13:38.266310 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:38.266267    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt" event={"ID":"191eb204-5730-4132-a3fc-4c3fa119f2c4","Type":"ContainerStarted","Data":"20b387d32ac484ef91c410e235a6c8f261fe91eb87d58da7640d3a696756b6f0"}
Sep 08 15:13:38.266692 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:38.266422    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:38.291177 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:38.291125    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt" podStartSLOduration=2.825773614 podStartE2EDuration="9.291108001s" totalImagesPullingTime="6.465334387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:13:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:13:38.289241751 +0000 UTC m=+266.375191796" watchObservedRunningTime="2026-09-08 15:13:38.291108001 +0000 UTC m=+266.377058042"
Sep 08 15:13:39.150151 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:39.150114    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:39.212207 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:39.212173    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:40.877261 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:40.877228    2575 patch_prober.go:28] interesting pod/image-registry-6c4d6bc854-bq6mc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 08 15:13:40.877638 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:40.877298    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc" podUID="17a6542b-38ff-4c74-aa24-cf8f5f3845a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 15:13:41.027903 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:41.027863    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:41.087821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:41.087788    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:42.285798 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:42.285764    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerStarted","Data":"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"}
Sep 08 15:13:42.286180 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:42.285888    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerStarted","Data":"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"}
Sep 08 15:13:42.286180 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:42.285936    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerStarted","Data":"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"}
Sep 08 15:13:42.286180 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:42.285977    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerStarted","Data":"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"}
Sep 08 15:13:43.162806 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:43.162767    2575 patch_prober.go:28] interesting pod/image-registry-6c4d6bc854-bq6mc container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 08 15:13:43.162973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:43.162821    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc" podUID="17a6542b-38ff-4c74-aa24-cf8f5f3845a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 15:13:43.292843 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:43.292805    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerStarted","Data":"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"}
Sep 08 15:13:43.293190 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:43.292883    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerStarted","Data":"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"}
Sep 08 15:13:43.324607 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:43.324441    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.162893696 podStartE2EDuration="10.324418875s" totalImagesPullingTime="7.137126998s" totalInitContainerRuntime="24.398181ms" isStatefulPod=false podCreationTimestamp="2026-09-08 15:13:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:13:43.3221552 +0000 UTC m=+271.408105244" watchObservedRunningTime="2026-09-08 15:13:43.324418875 +0000 UTC m=+271.410368918"
Sep 08 15:13:44.094992 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:44.094951    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:13:44.275787 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:44.275747    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5fcdbd7b65-dfmjt"
Sep 08 15:13:45.425696 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:45.425669    2575 scope.go:117] "RemoveContainer" containerID="15cf152da1cfb3fa741375982966ff8e6b3080dd0bf591f2ca8ee484d3882ff2"
Sep 08 15:13:46.305901 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:46.305872    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:13:46.306100 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:46.305925    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-nljdd" event={"ID":"3bf2739d-5bf1-43ee-a4a2-e2c75041d761","Type":"ContainerStarted","Data":"9b8fa126fc9783f79d804685f7d1c4fa449ba28db51eaaf9fac0a32b37620e03"}
Sep 08 15:13:46.306213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:46.306191    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:13:46.322834 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:46.322785    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-nljdd" podStartSLOduration=52.122254914 podStartE2EDuration="55.32277028s" totalImagesPullingTime="3.200515366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:13:46.321768125 +0000 UTC m=+274.407718168" watchObservedRunningTime="2026-09-08 15:13:46.32277028 +0000 UTC m=+274.408720323"
Sep 08 15:13:46.821480 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:46.821447    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-nljdd"
Sep 08 15:13:47.036162 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:47.036120    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:47.135915 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:47.135882    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:47.319909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:47.319871    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:47.684309 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:47.684256    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:47.784638 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:47.784596    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:48.077948 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:48.077862    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:48.143060 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:48.143022    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:48.498328 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:48.498288    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:48.868137 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:48.868111    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:49.092039 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:49.091994    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:50.311861 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:50.311818    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:50.876293 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:50.876261    2575 patch_prober.go:28] interesting pod/image-registry-6c4d6bc854-bq6mc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 08 15:13:50.876457 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:50.876313    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc" podUID="17a6542b-38ff-4c74-aa24-cf8f5f3845a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 15:13:52.170442 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:52.170392    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:52.170860 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:52.170468    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:13:53.163781 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:53.163744    2575 patch_prober.go:28] interesting pod/image-registry-6c4d6bc854-bq6mc container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 08 15:13:53.163942 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:53.163806    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc" podUID="17a6542b-38ff-4c74-aa24-cf8f5f3845a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 15:13:55.373653 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:55.373617    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:13:55.376039 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:55.376020    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f621c1ae-a9a4-4c7d-979c-adfbcafaf33a-metrics-tls\") pod \"dns-default-p4szr\" (UID: \"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a\") " pod="openshift-dns/dns-default-p4szr"
Sep 08 15:13:55.531749 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:55.531716    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cc4mw\""
Sep 08 15:13:55.539046 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:55.539027    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p4szr"
Sep 08 15:13:55.819003 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:55.818973    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p4szr"]
Sep 08 15:13:55.820973 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:13:55.820946    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf621c1ae_a9a4_4c7d_979c_adfbcafaf33a.slice/crio-b7862e80c653dd2eb3d8bb89fc18f988749e9a71db2b7088630e2691f8de96bf WatchSource:0}: Error finding container b7862e80c653dd2eb3d8bb89fc18f988749e9a71db2b7088630e2691f8de96bf: Status 404 returned error can't find the container with id b7862e80c653dd2eb3d8bb89fc18f988749e9a71db2b7088630e2691f8de96bf
Sep 08 15:13:56.346408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:56.346370    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p4szr" event={"ID":"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a","Type":"ContainerStarted","Data":"b7862e80c653dd2eb3d8bb89fc18f988749e9a71db2b7088630e2691f8de96bf"}
Sep 08 15:13:57.149808 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:57.149762    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:13:57.259645 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:13:57.259603    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:00.123136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:00.123091    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:00.259928 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:00.259889    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:00.363751 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:00.363713    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p4szr" event={"ID":"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a","Type":"ContainerStarted","Data":"98daae19c022a036eb668ca4ef534a0f72d2632ddff34e641649e87fb37469de"}
Sep 08 15:14:00.363941 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:00.363775    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p4szr" event={"ID":"f621c1ae-a9a4-4c7d-979c-adfbcafaf33a","Type":"ContainerStarted","Data":"5cc3838e990c6b15fd6c09c270f59f3f3ba83529162ada78d70371977b349cb7"}
Sep 08 15:14:00.363941 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:00.363928    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-p4szr"
Sep 08 15:14:00.384963 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:00.384850    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-p4szr" podStartSLOduration=251.620532634 podStartE2EDuration="4m15.384828281s" totalImagesPullingTime="3.764295647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:14:00.384459321 +0000 UTC m=+288.470409363" watchObservedRunningTime="2026-09-08 15:14:00.384828281 +0000 UTC m=+288.470778323"
Sep 08 15:14:00.876363 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:00.876328    2575 patch_prober.go:28] interesting pod/image-registry-6c4d6bc854-bq6mc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 08 15:14:00.876511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:00.876382    2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc" podUID="17a6542b-38ff-4c74-aa24-cf8f5f3845a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 15:14:00.876511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:00.876418    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:14:00.876983 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:00.876940    2575 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"868e3ce24cd5027b496a5ce4cce2be43e201d3f1986c734aa8dbc32be384eb37"} pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc" containerMessage="Container registry failed liveness probe, will be restarted"
Sep 08 15:14:00.880814 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:00.880790    2575 patch_prober.go:28] interesting pod/image-registry-6c4d6bc854-bq6mc container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 08 15:14:00.880930 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:00.880830    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc" podUID="17a6542b-38ff-4c74-aa24-cf8f5f3845a3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 15:14:01.370129 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:01.370087    2575 generic.go:356] "Generic (PLEG): container finished" podID="65032153-eabb-470d-ac9c-0490d906994f" containerID="b3d0040efa54e8782f0570c58552df560848580ae58a5d0b13d0c4a84d113584" exitCode=0
Sep 08 15:14:01.370551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:01.370168    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd" event={"ID":"65032153-eabb-470d-ac9c-0490d906994f","Type":"ContainerDied","Data":"b3d0040efa54e8782f0570c58552df560848580ae58a5d0b13d0c4a84d113584"}
Sep 08 15:14:01.370635 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:01.370614    2575 scope.go:117] "RemoveContainer" containerID="b3d0040efa54e8782f0570c58552df560848580ae58a5d0b13d0c4a84d113584"
Sep 08 15:14:02.376268 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:02.376231    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-8g8pd" event={"ID":"65032153-eabb-470d-ac9c-0490d906994f","Type":"ContainerStarted","Data":"3f8c3da247971c1675071fb6951f66428f4a7aacc3cd54320fc461904c5f98de"}
Sep 08 15:14:03.233160 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:03.233112    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p4szr_f621c1ae-a9a4-4c7d-979c-adfbcafaf33a/dns/0.log"
Sep 08 15:14:03.432730 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:03.432696    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p4szr_f621c1ae-a9a4-4c7d-979c-adfbcafaf33a/kube-rbac-proxy/0.log"
Sep 08 15:14:04.032123 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:04.032096    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4ltq9_d33f854f-48e2-45ca-8289-0f5cb0817be7/dns-node-resolver/0.log"
Sep 08 15:14:04.632633 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:04.632594    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c4d6bc854-bq6mc_17a6542b-38ff-4c74-aa24-cf8f5f3845a3/registry/0.log"
Sep 08 15:14:05.231795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:05.231767    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qvxxd_dea9369e-6c20-4b59-a04b-eb20d2f8cca5/node-ca/0.log"
Sep 08 15:14:06.032174 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:06.032142    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nvc47_a2e96079-8d75-4cd8-b77a-49f86c67a787/serve-healthcheck-canary/0.log"
Sep 08 15:14:06.359870 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:06.359830    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:08.233345 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:08.233305    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:08.935021 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:08.934986    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:09.037285 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:09.037243    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:10.372651 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:10.372620    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-p4szr"
Sep 08 15:14:10.881446 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:10.881417    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:14:11.058519 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:11.058361    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:11.169653 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:11.169569    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:12.175958 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:12.175930    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:14:12.179846 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:12.179819    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-565b9cbcb7-722mx"
Sep 08 15:14:12.282042 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:12.282010    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:14:12.283233 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:12.283205    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:14:12.295701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:12.295679    2575 kubelet.go:1648] "Image garbage collection succeeded"
Sep 08 15:14:13.265198 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:13.265158    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:13.328127 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:13.328092    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:15.115311 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:15.115273    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:15.175009 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:15.174971    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:17.012766 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:17.012734    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:18.520446 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:18.520409    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:18.621184 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:18.621145    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:19.513236 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:19.513200    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:19.613946 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:19.613914    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:21.543973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:21.543933    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:21.606172 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:21.606131    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:22.344092 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:22.344061    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:22.433391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:22.433356    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:22.731900 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:22.731859    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:22.831977 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:22.831931    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:25.896735 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:25.896695    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc" podUID="17a6542b-38ff-4c74-aa24-cf8f5f3845a3" containerName="registry" containerID="cri-o://868e3ce24cd5027b496a5ce4cce2be43e201d3f1986c734aa8dbc32be384eb37" gracePeriod=30
Sep 08 15:14:26.027424 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:26.027389    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:26.489571 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:26.489535    2575 generic.go:356] "Generic (PLEG): container finished" podID="742c48dd-0b32-418e-afb4-61f32954fbfd" containerID="d8ec0d0c62bdc943bf14dc8592492b7715ccc93959fcc28c232d9d5f11f2d543" exitCode=0
Sep 08 15:14:26.489742 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:26.489603    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk" event={"ID":"742c48dd-0b32-418e-afb4-61f32954fbfd","Type":"ContainerDied","Data":"d8ec0d0c62bdc943bf14dc8592492b7715ccc93959fcc28c232d9d5f11f2d543"}
Sep 08 15:14:26.489932 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:26.489918    2575 scope.go:117] "RemoveContainer" containerID="d8ec0d0c62bdc943bf14dc8592492b7715ccc93959fcc28c232d9d5f11f2d543"
Sep 08 15:14:27.495708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:27.495676    2575 generic.go:356] "Generic (PLEG): container finished" podID="17a6542b-38ff-4c74-aa24-cf8f5f3845a3" containerID="868e3ce24cd5027b496a5ce4cce2be43e201d3f1986c734aa8dbc32be384eb37" exitCode=0
Sep 08 15:14:27.496126 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:27.495761    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc" event={"ID":"17a6542b-38ff-4c74-aa24-cf8f5f3845a3","Type":"ContainerDied","Data":"868e3ce24cd5027b496a5ce4cce2be43e201d3f1986c734aa8dbc32be384eb37"}
Sep 08 15:14:27.496126 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:27.495818    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc" event={"ID":"17a6542b-38ff-4c74-aa24-cf8f5f3845a3","Type":"ContainerStarted","Data":"0be98f2e5b08c17573877b8fa640d52c628d6b976914de7fd2939dc7e5eb9ac9"}
Sep 08 15:14:27.496126 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:27.495844    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:14:27.497561 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:27.497539    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cwfvk" event={"ID":"742c48dd-0b32-418e-afb4-61f32954fbfd","Type":"ContainerStarted","Data":"3bceb07bb0f1891f93618cf21315bb6f4e192af7f487b5d9e06e7bb1131b26f5"}
Sep 08 15:14:27.866099 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:27.866046    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:27.967628 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:27.967594    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:33.144151 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:33.144116    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:33.245931 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:33.245895    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:33.357088 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:33.357045    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:34.094872 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:34.094829    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:34.115814 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:34.115786    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:34.539319 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:34.539280    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:41.051030 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:41.050992    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:41.151621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:41.151570    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:46.770510 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:46.770454    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:47.212704 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:47.212664    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:48.506723 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:48.506690    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c4d6bc854-bq6mc"
Sep 08 15:14:48.766091 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:48.765999    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:48.830559 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:48.830527    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:52.189551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.189517    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:14:52.189941 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.189800    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="prometheus" containerID="cri-o://b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0" gracePeriod=600
Sep 08 15:14:52.189941 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.189837    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="kube-rbac-proxy" containerID="cri-o://97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10" gracePeriod=600
Sep 08 15:14:52.189941 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.189840    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5" gracePeriod=600
Sep 08 15:14:52.189941 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.189866    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="thanos-sidecar" containerID="cri-o://b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf" gracePeriod=600
Sep 08 15:14:52.189941 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.189908    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="config-reloader" containerID="cri-o://898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b" gracePeriod=600
Sep 08 15:14:52.190170 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.189862    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="kube-rbac-proxy-web" containerID="cri-o://f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a" gracePeriod=600
Sep 08 15:14:52.428578 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.428529    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.471321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471248    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.471321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471290    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/523e70ee-482d-4851-999c-ff3d8b35f1b5-tls-assets\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.471321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471308    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-2\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.471625 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471344    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-serving-certs-ca-bundle\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.471625 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471369    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxfw2\" (UniqueName: \"kubernetes.io/projected/523e70ee-482d-4851-999c-ff3d8b35f1b5-kube-api-access-pxfw2\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.471625 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471408    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-db\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.471625 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471433    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-tls\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.471625 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471467    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-metrics-client-certs\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.471625 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471503    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-web-config\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.471625 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471530    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-grpc-tls\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.471625 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471554    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-thanos-prometheus-http-client-file\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.471625 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471585    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-metrics-client-ca\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.471625 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471628    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-1\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.472098 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471657    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-config\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.472098 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471682    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-kube-rbac-proxy\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.472098 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471723    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.472098 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471749    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/523e70ee-482d-4851-999c-ff3d8b35f1b5-config-out\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.472098 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471773    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-0\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.472098 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471799    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-trusted-ca-bundle\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.472098 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471829    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-kubelet-serving-ca-bundle\") pod \"523e70ee-482d-4851-999c-ff3d8b35f1b5\" (UID: \"523e70ee-482d-4851-999c-ff3d8b35f1b5\") "
Sep 08 15:14:52.472098 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.471833    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:14:52.472527 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.472135    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.472527 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.472467    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:14:52.472687 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.472668    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:14:52.473891 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.473022    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:14:52.473891 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.473577    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:14:52.474158 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.474088    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:14:52.475074 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.475041    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:14:52.475654 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.475625    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:14:52.476350 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.476311    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:14:52.477083 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.477042    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/523e70ee-482d-4851-999c-ff3d8b35f1b5-config-out" (OuterVolumeSpecName: "config-out") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:14:52.477603 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.477581    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/523e70ee-482d-4851-999c-ff3d8b35f1b5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:14:52.477981 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.477959    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:14:52.478082 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.478030    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:14:52.478149 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.478121    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:14:52.478414 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.478357    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-config" (OuterVolumeSpecName: "config") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:14:52.478414 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.478360    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/523e70ee-482d-4851-999c-ff3d8b35f1b5-kube-api-access-pxfw2" (OuterVolumeSpecName: "kube-api-access-pxfw2") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "kube-api-access-pxfw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:14:52.478584 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.478472    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:14:52.479478 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.479449    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:14:52.480051 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.480032    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:14:52.488896 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.488873    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-web-config" (OuterVolumeSpecName: "web-config") pod "523e70ee-482d-4851-999c-ff3d8b35f1b5" (UID: "523e70ee-482d-4851-999c-ff3d8b35f1b5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:14:52.573128 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573089    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573128 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573122    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/523e70ee-482d-4851-999c-ff3d8b35f1b5-tls-assets\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573128 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573134    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573146    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxfw2\" (UniqueName: \"kubernetes.io/projected/523e70ee-482d-4851-999c-ff3d8b35f1b5-kube-api-access-pxfw2\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573158    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-db\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573169    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573180    2575 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-metrics-client-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573191    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-web-config\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573202    2575 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-grpc-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573213    2575 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573225    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-metrics-client-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573236    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573248    2575 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-config\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573260    2575 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-kube-rbac-proxy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573272    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/523e70ee-482d-4851-999c-ff3d8b35f1b5-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573283    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/523e70ee-482d-4851-999c-ff3d8b35f1b5-config-out\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573295    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573307    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.573390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.573319    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/523e70ee-482d-4851-999c-ff3d8b35f1b5-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:14:52.598308 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598276    2575 generic.go:356] "Generic (PLEG): container finished" podID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerID="3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5" exitCode=0
Sep 08 15:14:52.598308 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598299    2575 generic.go:356] "Generic (PLEG): container finished" podID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerID="97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10" exitCode=0
Sep 08 15:14:52.598308 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598305    2575 generic.go:356] "Generic (PLEG): container finished" podID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerID="f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a" exitCode=0
Sep 08 15:14:52.598308 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598313    2575 generic.go:356] "Generic (PLEG): container finished" podID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerID="b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf" exitCode=0
Sep 08 15:14:52.598569 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598318    2575 generic.go:356] "Generic (PLEG): container finished" podID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerID="898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b" exitCode=0
Sep 08 15:14:52.598569 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598323    2575 generic.go:356] "Generic (PLEG): container finished" podID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerID="b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0" exitCode=0
Sep 08 15:14:52.598569 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598344    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.598569 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598349    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerDied","Data":"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"}
Sep 08 15:14:52.598569 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598415    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerDied","Data":"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"}
Sep 08 15:14:52.598569 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598442    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerDied","Data":"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"}
Sep 08 15:14:52.598569 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598467    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerDied","Data":"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"}
Sep 08 15:14:52.598569 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598483    2575 scope.go:117] "RemoveContainer" containerID="3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"
Sep 08 15:14:52.598569 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598503    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerDied","Data":"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"}
Sep 08 15:14:52.598569 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598532    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerDied","Data":"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"}
Sep 08 15:14:52.598569 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.598555    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"523e70ee-482d-4851-999c-ff3d8b35f1b5","Type":"ContainerDied","Data":"ca53689810fe906bcff7f0217bf6e180ddc9735993c447bb79d0c3348609cc26"}
Sep 08 15:14:52.607183 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.607131    2575 scope.go:117] "RemoveContainer" containerID="97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"
Sep 08 15:14:52.615004 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.614986    2575 scope.go:117] "RemoveContainer" containerID="f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"
Sep 08 15:14:52.621374 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.621360    2575 scope.go:117] "RemoveContainer" containerID="b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"
Sep 08 15:14:52.624937 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.624915    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:14:52.628483 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.628465    2575 scope.go:117] "RemoveContainer" containerID="898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"
Sep 08 15:14:52.631212 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.631188    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:14:52.635501 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.635474    2575 scope.go:117] "RemoveContainer" containerID="b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"
Sep 08 15:14:52.642373 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.642353    2575 scope.go:117] "RemoveContainer" containerID="ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"
Sep 08 15:14:52.648761 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.648743    2575 scope.go:117] "RemoveContainer" containerID="3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"
Sep 08 15:14:52.649008 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:14:52.648990    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5\": container with ID starting with 3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5 not found: ID does not exist" containerID="3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"
Sep 08 15:14:52.649052 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.649015    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"} err="failed to get container status \"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5\": rpc error: code = NotFound desc = could not find container \"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5\": container with ID starting with 3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5 not found: ID does not exist"
Sep 08 15:14:52.649052 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.649031    2575 scope.go:117] "RemoveContainer" containerID="97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"
Sep 08 15:14:52.649275 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:14:52.649255    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10\": container with ID starting with 97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10 not found: ID does not exist" containerID="97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"
Sep 08 15:14:52.649326 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.649278    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"} err="failed to get container status \"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10\": rpc error: code = NotFound desc = could not find container \"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10\": container with ID starting with 97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10 not found: ID does not exist"
Sep 08 15:14:52.649326 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.649291    2575 scope.go:117] "RemoveContainer" containerID="f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"
Sep 08 15:14:52.649486 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:14:52.649470    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a\": container with ID starting with f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a not found: ID does not exist" containerID="f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"
Sep 08 15:14:52.649546 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.649503    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"} err="failed to get container status \"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a\": rpc error: code = NotFound desc = could not find container \"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a\": container with ID starting with f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a not found: ID does not exist"
Sep 08 15:14:52.649546 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.649517    2575 scope.go:117] "RemoveContainer" containerID="b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"
Sep 08 15:14:52.649820 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:14:52.649794    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf\": container with ID starting with b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf not found: ID does not exist" containerID="b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"
Sep 08 15:14:52.649913 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.649826    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"} err="failed to get container status \"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf\": rpc error: code = NotFound desc = could not find container \"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf\": container with ID starting with b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf not found: ID does not exist"
Sep 08 15:14:52.649913 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.649849    2575 scope.go:117] "RemoveContainer" containerID="898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"
Sep 08 15:14:52.650116 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:14:52.650098    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b\": container with ID starting with 898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b not found: ID does not exist" containerID="898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"
Sep 08 15:14:52.650184 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.650124    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"} err="failed to get container status \"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b\": rpc error: code = NotFound desc = could not find container \"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b\": container with ID starting with 898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b not found: ID does not exist"
Sep 08 15:14:52.650184 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.650145    2575 scope.go:117] "RemoveContainer" containerID="b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"
Sep 08 15:14:52.650399 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:14:52.650372    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0\": container with ID starting with b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0 not found: ID does not exist" containerID="b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"
Sep 08 15:14:52.650511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.650402    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"} err="failed to get container status \"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0\": rpc error: code = NotFound desc = could not find container \"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0\": container with ID starting with b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0 not found: ID does not exist"
Sep 08 15:14:52.650511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.650416    2575 scope.go:117] "RemoveContainer" containerID="ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"
Sep 08 15:14:52.650688 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:14:52.650670    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06\": container with ID starting with ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06 not found: ID does not exist" containerID="ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"
Sep 08 15:14:52.650725 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.650692    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"} err="failed to get container status \"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06\": rpc error: code = NotFound desc = could not find container \"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06\": container with ID starting with ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06 not found: ID does not exist"
Sep 08 15:14:52.650725 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.650705    2575 scope.go:117] "RemoveContainer" containerID="3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"
Sep 08 15:14:52.651028 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.651011    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"} err="failed to get container status \"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5\": rpc error: code = NotFound desc = could not find container \"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5\": container with ID starting with 3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5 not found: ID does not exist"
Sep 08 15:14:52.651028 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.651027    2575 scope.go:117] "RemoveContainer" containerID="97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"
Sep 08 15:14:52.651286 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.651267    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"} err="failed to get container status \"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10\": rpc error: code = NotFound desc = could not find container \"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10\": container with ID starting with 97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10 not found: ID does not exist"
Sep 08 15:14:52.651334 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.651294    2575 scope.go:117] "RemoveContainer" containerID="f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"
Sep 08 15:14:52.651543 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.651525    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"} err="failed to get container status \"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a\": rpc error: code = NotFound desc = could not find container \"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a\": container with ID starting with f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a not found: ID does not exist"
Sep 08 15:14:52.651605 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.651544    2575 scope.go:117] "RemoveContainer" containerID="b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"
Sep 08 15:14:52.651828 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.651808    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"} err="failed to get container status \"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf\": rpc error: code = NotFound desc = could not find container \"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf\": container with ID starting with b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf not found: ID does not exist"
Sep 08 15:14:52.651889 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.651828    2575 scope.go:117] "RemoveContainer" containerID="898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"
Sep 08 15:14:52.652045 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.652030    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"} err="failed to get container status \"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b\": rpc error: code = NotFound desc = could not find container \"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b\": container with ID starting with 898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b not found: ID does not exist"
Sep 08 15:14:52.652093 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.652044    2575 scope.go:117] "RemoveContainer" containerID="b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"
Sep 08 15:14:52.652283 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.652264    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"} err="failed to get container status \"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0\": rpc error: code = NotFound desc = could not find container \"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0\": container with ID starting with b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0 not found: ID does not exist"
Sep 08 15:14:52.652356 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.652284    2575 scope.go:117] "RemoveContainer" containerID="ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"
Sep 08 15:14:52.652549 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.652486    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"} err="failed to get container status \"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06\": rpc error: code = NotFound desc = could not find container \"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06\": container with ID starting with ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06 not found: ID does not exist"
Sep 08 15:14:52.652549 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.652546    2575 scope.go:117] "RemoveContainer" containerID="3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"
Sep 08 15:14:52.652793 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.652772    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"} err="failed to get container status \"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5\": rpc error: code = NotFound desc = could not find container \"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5\": container with ID starting with 3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5 not found: ID does not exist"
Sep 08 15:14:52.652862 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.652794    2575 scope.go:117] "RemoveContainer" containerID="97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"
Sep 08 15:14:52.653016 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.652997    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"} err="failed to get container status \"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10\": rpc error: code = NotFound desc = could not find container \"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10\": container with ID starting with 97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10 not found: ID does not exist"
Sep 08 15:14:52.653088 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.653018    2575 scope.go:117] "RemoveContainer" containerID="f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"
Sep 08 15:14:52.653238 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.653221    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"} err="failed to get container status \"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a\": rpc error: code = NotFound desc = could not find container \"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a\": container with ID starting with f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a not found: ID does not exist"
Sep 08 15:14:52.653285 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.653238    2575 scope.go:117] "RemoveContainer" containerID="b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"
Sep 08 15:14:52.653448 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.653429    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"} err="failed to get container status \"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf\": rpc error: code = NotFound desc = could not find container \"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf\": container with ID starting with b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf not found: ID does not exist"
Sep 08 15:14:52.653526 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.653449    2575 scope.go:117] "RemoveContainer" containerID="898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"
Sep 08 15:14:52.653663 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.653648    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"} err="failed to get container status \"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b\": rpc error: code = NotFound desc = could not find container \"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b\": container with ID starting with 898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b not found: ID does not exist"
Sep 08 15:14:52.653706 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.653663    2575 scope.go:117] "RemoveContainer" containerID="b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"
Sep 08 15:14:52.653869 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.653851    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"} err="failed to get container status \"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0\": rpc error: code = NotFound desc = could not find container \"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0\": container with ID starting with b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0 not found: ID does not exist"
Sep 08 15:14:52.653934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.653870    2575 scope.go:117] "RemoveContainer" containerID="ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"
Sep 08 15:14:52.654070 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.654053    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"} err="failed to get container status \"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06\": rpc error: code = NotFound desc = could not find container \"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06\": container with ID starting with ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06 not found: ID does not exist"
Sep 08 15:14:52.654070 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.654068    2575 scope.go:117] "RemoveContainer" containerID="3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"
Sep 08 15:14:52.654251 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.654237    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"} err="failed to get container status \"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5\": rpc error: code = NotFound desc = could not find container \"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5\": container with ID starting with 3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5 not found: ID does not exist"
Sep 08 15:14:52.654290 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.654252    2575 scope.go:117] "RemoveContainer" containerID="97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"
Sep 08 15:14:52.654449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.654430    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"} err="failed to get container status \"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10\": rpc error: code = NotFound desc = could not find container \"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10\": container with ID starting with 97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10 not found: ID does not exist"
Sep 08 15:14:52.654529 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.654449    2575 scope.go:117] "RemoveContainer" containerID="f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"
Sep 08 15:14:52.654664 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.654648    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"} err="failed to get container status \"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a\": rpc error: code = NotFound desc = could not find container \"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a\": container with ID starting with f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a not found: ID does not exist"
Sep 08 15:14:52.654742 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.654665    2575 scope.go:117] "RemoveContainer" containerID="b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"
Sep 08 15:14:52.654854 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.654837    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"} err="failed to get container status \"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf\": rpc error: code = NotFound desc = could not find container \"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf\": container with ID starting with b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf not found: ID does not exist"
Sep 08 15:14:52.654902 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.654855    2575 scope.go:117] "RemoveContainer" containerID="898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"
Sep 08 15:14:52.655058 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.655043    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"} err="failed to get container status \"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b\": rpc error: code = NotFound desc = could not find container \"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b\": container with ID starting with 898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b not found: ID does not exist"
Sep 08 15:14:52.655102 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.655058    2575 scope.go:117] "RemoveContainer" containerID="b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"
Sep 08 15:14:52.655232 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.655217    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"} err="failed to get container status \"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0\": rpc error: code = NotFound desc = could not find container \"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0\": container with ID starting with b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0 not found: ID does not exist"
Sep 08 15:14:52.655282 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.655231    2575 scope.go:117] "RemoveContainer" containerID="ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"
Sep 08 15:14:52.655442 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.655424    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"} err="failed to get container status \"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06\": rpc error: code = NotFound desc = could not find container \"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06\": container with ID starting with ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06 not found: ID does not exist"
Sep 08 15:14:52.655530 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.655444    2575 scope.go:117] "RemoveContainer" containerID="3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"
Sep 08 15:14:52.655667 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.655652    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"} err="failed to get container status \"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5\": rpc error: code = NotFound desc = could not find container \"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5\": container with ID starting with 3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5 not found: ID does not exist"
Sep 08 15:14:52.655707 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.655667    2575 scope.go:117] "RemoveContainer" containerID="97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"
Sep 08 15:14:52.655850 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.655831    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"} err="failed to get container status \"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10\": rpc error: code = NotFound desc = could not find container \"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10\": container with ID starting with 97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10 not found: ID does not exist"
Sep 08 15:14:52.655918 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.655851    2575 scope.go:117] "RemoveContainer" containerID="f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"
Sep 08 15:14:52.656059 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.656041    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"} err="failed to get container status \"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a\": rpc error: code = NotFound desc = could not find container \"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a\": container with ID starting with f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a not found: ID does not exist"
Sep 08 15:14:52.656104 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.656061    2575 scope.go:117] "RemoveContainer" containerID="b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"
Sep 08 15:14:52.656257 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.656239    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"} err="failed to get container status \"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf\": rpc error: code = NotFound desc = could not find container \"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf\": container with ID starting with b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf not found: ID does not exist"
Sep 08 15:14:52.656315 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.656258    2575 scope.go:117] "RemoveContainer" containerID="898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"
Sep 08 15:14:52.656459 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.656439    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"} err="failed to get container status \"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b\": rpc error: code = NotFound desc = could not find container \"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b\": container with ID starting with 898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b not found: ID does not exist"
Sep 08 15:14:52.656575 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.656459    2575 scope.go:117] "RemoveContainer" containerID="b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"
Sep 08 15:14:52.656726 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.656707    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"} err="failed to get container status \"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0\": rpc error: code = NotFound desc = could not find container \"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0\": container with ID starting with b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0 not found: ID does not exist"
Sep 08 15:14:52.656793 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.656727    2575 scope.go:117] "RemoveContainer" containerID="ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"
Sep 08 15:14:52.656936 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.656920    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"} err="failed to get container status \"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06\": rpc error: code = NotFound desc = could not find container \"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06\": container with ID starting with ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06 not found: ID does not exist"
Sep 08 15:14:52.656980 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.656936    2575 scope.go:117] "RemoveContainer" containerID="3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"
Sep 08 15:14:52.657124 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.657107    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5"} err="failed to get container status \"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5\": rpc error: code = NotFound desc = could not find container \"3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5\": container with ID starting with 3682dd964514ef41645a60c87503d749f6fe06ba963cc5c92120ee9d737438c5 not found: ID does not exist"
Sep 08 15:14:52.657183 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.657125    2575 scope.go:117] "RemoveContainer" containerID="97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"
Sep 08 15:14:52.657316 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.657297    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10"} err="failed to get container status \"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10\": rpc error: code = NotFound desc = could not find container \"97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10\": container with ID starting with 97cef304bd4648dd41a4ba35ec3eb74e96e0b8bba7923c1e110d0588e19c8b10 not found: ID does not exist"
Sep 08 15:14:52.657360 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.657316    2575 scope.go:117] "RemoveContainer" containerID="f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"
Sep 08 15:14:52.657514 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.657473    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a"} err="failed to get container status \"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a\": rpc error: code = NotFound desc = could not find container \"f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a\": container with ID starting with f898f002f10b7c1959d2ab0fb9117ba40a8c0ccf30b46f945ddc1dfc55450d4a not found: ID does not exist"
Sep 08 15:14:52.657570 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.657514    2575 scope.go:117] "RemoveContainer" containerID="b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"
Sep 08 15:14:52.657712 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.657691    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf"} err="failed to get container status \"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf\": rpc error: code = NotFound desc = could not find container \"b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf\": container with ID starting with b71aabe8ab26c24d2cc2f8a28601d6c1ca3f73e73a4bebf802c79610c284d5bf not found: ID does not exist"
Sep 08 15:14:52.657750 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.657712    2575 scope.go:117] "RemoveContainer" containerID="898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"
Sep 08 15:14:52.657945 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.657928    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b"} err="failed to get container status \"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b\": rpc error: code = NotFound desc = could not find container \"898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b\": container with ID starting with 898986d20bbe33ddbe1194c76e2e7ea38d1a22a178d51054561c8bf374c1b57b not found: ID does not exist"
Sep 08 15:14:52.657945 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.657944    2575 scope.go:117] "RemoveContainer" containerID="b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"
Sep 08 15:14:52.658213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.658189    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0"} err="failed to get container status \"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0\": rpc error: code = NotFound desc = could not find container \"b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0\": container with ID starting with b159522dce36dca9a9a3c36f83b2a9b132319c66fb40fd612a55f71815fa4cd0 not found: ID does not exist"
Sep 08 15:14:52.658313 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.658215    2575 scope.go:117] "RemoveContainer" containerID="ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"
Sep 08 15:14:52.658663 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.658632    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06"} err="failed to get container status \"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06\": rpc error: code = NotFound desc = could not find container \"ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06\": container with ID starting with ba7517cf8d2cf2e2e771466c9f6ad890493f95b1f289ce4d5ef005d228d5cd06 not found: ID does not exist"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.660556    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.660948    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="thanos-sidecar"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.660962    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="thanos-sidecar"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.660975    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="kube-rbac-proxy-thanos"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.660984    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="kube-rbac-proxy-thanos"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.660994    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="kube-rbac-proxy-web"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661003    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="kube-rbac-proxy-web"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661013    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="init-config-reloader"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661021    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="init-config-reloader"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661048    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="config-reloader"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661056    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="config-reloader"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661072    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="kube-rbac-proxy"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661080    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="kube-rbac-proxy"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661090    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="prometheus"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661098    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="prometheus"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661160    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="kube-rbac-proxy-thanos"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661174    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="prometheus"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661182    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="config-reloader"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661191    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="kube-rbac-proxy"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661198    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="kube-rbac-proxy-web"
Sep 08 15:14:52.662849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.661208    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" containerName="thanos-sidecar"
Sep 08 15:14:52.666588 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.666570    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.669547 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.669521    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 08 15:14:52.669719 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.669704    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 08 15:14:52.669826 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.669790    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 08 15:14:52.669941 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.669896    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 08 15:14:52.670053 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.670035    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 08 15:14:52.670053 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.670047    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 08 15:14:52.670053 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.670058    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 08 15:14:52.670246 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.670079    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p9rz7\""
Sep 08 15:14:52.670246 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.670151    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 08 15:14:52.670246 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.670156    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 08 15:14:52.670246 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.670199    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d8nh4eg2s042j\""
Sep 08 15:14:52.670525 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.670510    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 08 15:14:52.670591 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.670562    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 08 15:14:52.670591 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.670580    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 08 15:14:52.675737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.675696    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 08 15:14:52.679567 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.679543    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 08 15:14:52.684263 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.684243    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:14:52.775157 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775067    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4aaf26c4-7510-495c-b25e-3d47fe66718a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775157 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775108    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775157 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775129    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775452 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775237    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775452 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775274    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775452 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775321    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775452 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775348    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775452 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775406    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775452 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775431    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-web-config\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775454    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775484    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775535    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775558    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775654    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775682    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-config\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775702    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775717    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775990 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775764    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775990 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775788    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4aaf26c4-7510-495c-b25e-3d47fe66718a-config-out\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.775990 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.775809    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsl9k\" (UniqueName: \"kubernetes.io/projected/4aaf26c4-7510-495c-b25e-3d47fe66718a-kube-api-access-vsl9k\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.876466 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876420    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4aaf26c4-7510-495c-b25e-3d47fe66718a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.876466 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876470    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.876744 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876525    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.876744 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876562    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.876744 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876688    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.876744 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876739    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.876943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876769    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.876943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876817    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.876943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876842    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-web-config\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.876943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876866    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.876943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876898    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.876943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876924    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.877208 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876949    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.877208 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.876993    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.877208 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.877025    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-config\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.877208 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.877053    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.877208 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.877077    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.877208 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.877106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.877208 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.877134    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4aaf26c4-7510-495c-b25e-3d47fe66718a-config-out\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.877208 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.877179    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsl9k\" (UniqueName: \"kubernetes.io/projected/4aaf26c4-7510-495c-b25e-3d47fe66718a-kube-api-access-vsl9k\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.877634 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.877403    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.877634 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.877466    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.878281 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.877968    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.878281 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.878255    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.879728 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.879704    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4aaf26c4-7510-495c-b25e-3d47fe66718a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.879947 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.879918    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.880094 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.877412    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.880192 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.880152    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.880378 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.880350    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.881278 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.880883    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-web-config\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.881278 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.881196    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.881278 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.881208    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.881278 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.881247    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.881278 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.881267    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4aaf26c4-7510-495c-b25e-3d47fe66718a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.881570 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.881318    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-config\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.882128 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.882107    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.882421 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.882402    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.882470 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.882406    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4aaf26c4-7510-495c-b25e-3d47fe66718a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.883028 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.883010    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4aaf26c4-7510-495c-b25e-3d47fe66718a-config-out\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.885939 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.885916    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsl9k\" (UniqueName: \"kubernetes.io/projected/4aaf26c4-7510-495c-b25e-3d47fe66718a-kube-api-access-vsl9k\") pod \"prometheus-k8s-0\" (UID: \"4aaf26c4-7510-495c-b25e-3d47fe66718a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:52.981678 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:52.981639    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:14:53.264102 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:53.264069    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:14:53.267372 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:14:53.267345    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aaf26c4_7510_495c_b25e_3d47fe66718a.slice/crio-643dd5d122a4e1f109b1fee4324a30286dd81bdc61d1990f3aca16bdeaa2dd9c WatchSource:0}: Error finding container 643dd5d122a4e1f109b1fee4324a30286dd81bdc61d1990f3aca16bdeaa2dd9c: Status 404 returned error can't find the container with id 643dd5d122a4e1f109b1fee4324a30286dd81bdc61d1990f3aca16bdeaa2dd9c
Sep 08 15:14:53.605965 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:53.605936    2575 generic.go:356] "Generic (PLEG): container finished" podID="4aaf26c4-7510-495c-b25e-3d47fe66718a" containerID="cd3ecab7aa0fe5199370be98f767d2f3752ddb22a38080b9da4291b55666185b" exitCode=0
Sep 08 15:14:53.606106 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:53.606005    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4aaf26c4-7510-495c-b25e-3d47fe66718a","Type":"ContainerDied","Data":"cd3ecab7aa0fe5199370be98f767d2f3752ddb22a38080b9da4291b55666185b"}
Sep 08 15:14:53.606106 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:53.606058    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4aaf26c4-7510-495c-b25e-3d47fe66718a","Type":"ContainerStarted","Data":"643dd5d122a4e1f109b1fee4324a30286dd81bdc61d1990f3aca16bdeaa2dd9c"}
Sep 08 15:14:54.432024 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:54.431993    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="523e70ee-482d-4851-999c-ff3d8b35f1b5" path="/var/lib/kubelet/pods/523e70ee-482d-4851-999c-ff3d8b35f1b5/volumes"
Sep 08 15:14:54.613795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:54.613766    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4aaf26c4-7510-495c-b25e-3d47fe66718a","Type":"ContainerStarted","Data":"fbea6caa338c6ddb077a1b776bbbf2e7c55d4e197a3af88c1047d0aabec57356"}
Sep 08 15:14:54.613936 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:54.613825    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4aaf26c4-7510-495c-b25e-3d47fe66718a","Type":"ContainerStarted","Data":"f47ec65d899061766e04c5c3a5530343f76f4570754af7f0ed56a2eedab1dc9e"}
Sep 08 15:14:54.613936 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:54.613853    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4aaf26c4-7510-495c-b25e-3d47fe66718a","Type":"ContainerStarted","Data":"1fa62f3b2a8d2a0a399748bded4cf0fdf14d0b46370cc0849be23c96f15d609b"}
Sep 08 15:14:54.613936 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:54.613878    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4aaf26c4-7510-495c-b25e-3d47fe66718a","Type":"ContainerStarted","Data":"0e4053e454c035911d79eb168982616bbc277ed68d8c33d5fcfe0063b3d3b983"}
Sep 08 15:14:54.613936 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:54.613904    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4aaf26c4-7510-495c-b25e-3d47fe66718a","Type":"ContainerStarted","Data":"55250636226b279a50fc9a1d09e515daa829c9f55c76657dc1cf14dcee898921"}
Sep 08 15:14:54.613936 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:54.613928    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4aaf26c4-7510-495c-b25e-3d47fe66718a","Type":"ContainerStarted","Data":"c3e346fd184e1d8db24f2f9393855b01df9171e71f0565f9be7088094489163e"}
Sep 08 15:14:54.647370 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:54.647310    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.628120601 podStartE2EDuration="2.647288616s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.168015ms" isStatefulPod=false podCreationTimestamp="2026-09-08 15:14:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:14:54.646399654 +0000 UTC m=+342.732349711" watchObservedRunningTime="2026-09-08 15:14:54.647288616 +0000 UTC m=+342.733238667"
Sep 08 15:14:56.063889 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:56.063847    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:56.164973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:56.164930    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:56.376265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:56.376231    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:56.432834 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:56.432802    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:56.517249 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:56.517215    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:56.585284 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:56.585242    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:57.164507 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:57.164471    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:57.263825 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:57.263787    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:14:57.982796 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:14:57.982762    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:15:07.400696 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:15:07.400610    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:15:07.501153 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:15:07.501116    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:15:07.600845 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:15:07.600802    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:15:11.033177 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:15:11.033139    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:15:11.112123 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:15:11.112084    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:15:32.469682 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:15:32.469647    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:15:52.982719 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:15:52.982685    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:15:52.997201 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:15:52.997176    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:15:53.861024 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:15:53.860998    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:16:02.024653 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:02.024616    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:02.092348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:02.092300    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:10.496284 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:10.496248    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:10.600390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:10.600356    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:10.660985 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:10.660947    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:11.196161 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:11.196120    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:11.297861 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:11.297827    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:13.941907 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:13.941862    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:13.994863 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:13.994833    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:21.697125 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:21.697085    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:21.758701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:21.758669    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:21.829523 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:21.829478    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:26.034704 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:26.034663    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:26.135650 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:26.135620    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:16:26.376402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:16:26.376379    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:06.418087 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:06.418050    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:06.536364 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:06.536334    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:43.870068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:43.870032    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:47.264260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:47.264225    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:47.356854 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:47.356809    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:47.561418 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:47.561348    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:48.620515 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:48.620473    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:51.822644 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:51.822605    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:51.925219 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:51.925184    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:52.968454 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:52.968423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:53.182868 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:53.182831    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:53.567549 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:53.567516    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:53.756160 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:53.756131    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:55.528165 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:55.528117    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:55.727933 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:55.727899    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:58.312044 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:58.312009    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:17:58.375550 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:17:58.375523    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:02.738051 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:02.738015    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:04.445837 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:04.445802    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:05.720853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:05.720820    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:05.825860 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:05.825827    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:13.919824 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:13.919783    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:14.845535 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:14.845504    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:14.931586 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:14.931552    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:15.487918 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:15.487889    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:16.569115 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:16.569084    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:18.951268 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:18.951192    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:19.068794 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:19.068765    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:27.774135 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:27.774099    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:27.880583 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:27.880552    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:28.388759 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:28.388728    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:28.785542 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:28.785470    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:28.905361 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:28.905328    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:29.012891 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.012863    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"]
Sep 08 15:18:29.016342 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.016327    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.020227 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.020192    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jgmwt\""
Sep 08 15:18:29.020378 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.020355    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 08 15:18:29.020452 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.020378    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 08 15:18:29.020452 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.020420    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 08 15:18:29.023620 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.023604    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 08 15:18:29.064480 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.064426    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"]
Sep 08 15:18:29.089806 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.089783    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-manifests\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.089883 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.089813    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2824g\" (UniqueName: \"kubernetes.io/projected/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-kube-api-access-2824g\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.089883 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.089862    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.089953 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.089916    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-tmp\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.089953 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.089933    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-webhook-cert\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.190832 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.190804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.190939 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.190860    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-tmp\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.190939 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.190877    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-webhook-cert\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.190939 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.190907    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-manifests\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.190939 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.190923    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2824g\" (UniqueName: \"kubernetes.io/projected/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-kube-api-access-2824g\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.191334 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.191316    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-manifests\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.193159 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.193131    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-tmp\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.193254 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.193176    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-webhook-cert\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.193296 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.193273    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.207874 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.207850    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2824g\" (UniqueName: \"kubernetes.io/projected/ff469016-0608-4fa7-bd5b-2ffff8ca88f7-kube-api-access-2824g\") pod \"opendatahub-operator-controller-manager-66db8bdb84-8j75r\" (UID: \"ff469016-0608-4fa7-bd5b-2ffff8ca88f7\") " pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.232032 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.232005    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:29.325477 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.325425    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:29.329265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.329219    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:29.430241 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.430211    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:29.533393 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.533360    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:29.605605 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.605584    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"]
Sep 08 15:18:29.608334 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:18:29.608313    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff469016_0608_4fa7_bd5b_2ffff8ca88f7.slice/crio-608b8f614337f42bcac2add83d6441cb424f1b8f0a053bcc98c3924425d2cd7c WatchSource:0}: Error finding container 608b8f614337f42bcac2add83d6441cb424f1b8f0a053bcc98c3924425d2cd7c: Status 404 returned error can't find the container with id 608b8f614337f42bcac2add83d6441cb424f1b8f0a053bcc98c3924425d2cd7c
Sep 08 15:18:29.610072 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.610053    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:18:29.634449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.634414    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:29.732568 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.732544    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:29.831318 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:29.831269    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:30.441966 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:30.441919    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r" event={"ID":"ff469016-0608-4fa7-bd5b-2ffff8ca88f7","Type":"ContainerStarted","Data":"608b8f614337f42bcac2add83d6441cb424f1b8f0a053bcc98c3924425d2cd7c"}
Sep 08 15:18:33.456177 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:33.456144    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r" event={"ID":"ff469016-0608-4fa7-bd5b-2ffff8ca88f7","Type":"ContainerStarted","Data":"31bd3a04a69fac29c00375fdddeba0c3475872c2b3527dd043e6f8f6ba616bf2"}
Sep 08 15:18:34.461706 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:34.461672    2575 generic.go:356] "Generic (PLEG): container finished" podID="ff469016-0608-4fa7-bd5b-2ffff8ca88f7" containerID="31bd3a04a69fac29c00375fdddeba0c3475872c2b3527dd043e6f8f6ba616bf2" exitCode=0
Sep 08 15:18:34.462042 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:34.461740    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r" event={"ID":"ff469016-0608-4fa7-bd5b-2ffff8ca88f7","Type":"ContainerDied","Data":"31bd3a04a69fac29c00375fdddeba0c3475872c2b3527dd043e6f8f6ba616bf2"}
Sep 08 15:18:35.468143 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:35.468112    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r" event={"ID":"ff469016-0608-4fa7-bd5b-2ffff8ca88f7","Type":"ContainerStarted","Data":"411fe525affd2a7713e75b77def20719445baafd5e0a3b3e6e90d02f4422d6f8"}
Sep 08 15:18:35.468468 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:35.468344    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:35.498239 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:35.498198    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r" podStartSLOduration=3.135323197 podStartE2EDuration="7.498185228s" totalImagesPullingTime="3.315244574s" totalInitContainerRuntime="1.047617457s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:35.497140525 +0000 UTC m=+563.583090568" watchObservedRunningTime="2026-09-08 15:18:35.498185228 +0000 UTC m=+563.584135269"
Sep 08 15:18:44.791279 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:44.791237    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:44.892886 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:44.892860    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:46.473597 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.473568    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66db8bdb84-8j75r"
Sep 08 15:18:46.778405 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.778334    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"]
Sep 08 15:18:46.788747 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.788725    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:46.789746 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.789722    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"]
Sep 08 15:18:46.791358 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.791338    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 08 15:18:46.791449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.791374    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 08 15:18:46.791723 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.791542    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 08 15:18:46.791873 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.791851    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mkrrt\""
Sep 08 15:18:46.791873 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.791869    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 08 15:18:46.792038 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.791902    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:18:46.823663 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.823645    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9215463-91d7-47fa-849b-57a626693cb1-cert\") pod \"lws-controller-manager-7f7779588f-hkxc7\" (UID: \"a9215463-91d7-47fa-849b-57a626693cb1\") " pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:46.823755 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.823683    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a9215463-91d7-47fa-849b-57a626693cb1-manager-config\") pod \"lws-controller-manager-7f7779588f-hkxc7\" (UID: \"a9215463-91d7-47fa-849b-57a626693cb1\") " pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:46.823755 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.823700    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlxcv\" (UniqueName: \"kubernetes.io/projected/a9215463-91d7-47fa-849b-57a626693cb1-kube-api-access-jlxcv\") pod \"lws-controller-manager-7f7779588f-hkxc7\" (UID: \"a9215463-91d7-47fa-849b-57a626693cb1\") " pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:46.823755 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.823738    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a9215463-91d7-47fa-849b-57a626693cb1-metrics-cert\") pod \"lws-controller-manager-7f7779588f-hkxc7\" (UID: \"a9215463-91d7-47fa-849b-57a626693cb1\") " pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:46.924677 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.924659    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9215463-91d7-47fa-849b-57a626693cb1-cert\") pod \"lws-controller-manager-7f7779588f-hkxc7\" (UID: \"a9215463-91d7-47fa-849b-57a626693cb1\") " pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:46.924761 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.924694    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a9215463-91d7-47fa-849b-57a626693cb1-manager-config\") pod \"lws-controller-manager-7f7779588f-hkxc7\" (UID: \"a9215463-91d7-47fa-849b-57a626693cb1\") " pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:46.924761 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.924711    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlxcv\" (UniqueName: \"kubernetes.io/projected/a9215463-91d7-47fa-849b-57a626693cb1-kube-api-access-jlxcv\") pod \"lws-controller-manager-7f7779588f-hkxc7\" (UID: \"a9215463-91d7-47fa-849b-57a626693cb1\") " pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:46.924761 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.924731    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a9215463-91d7-47fa-849b-57a626693cb1-metrics-cert\") pod \"lws-controller-manager-7f7779588f-hkxc7\" (UID: \"a9215463-91d7-47fa-849b-57a626693cb1\") " pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:46.925295 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.925275    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a9215463-91d7-47fa-849b-57a626693cb1-manager-config\") pod \"lws-controller-manager-7f7779588f-hkxc7\" (UID: \"a9215463-91d7-47fa-849b-57a626693cb1\") " pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:46.926906 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.926885    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9215463-91d7-47fa-849b-57a626693cb1-cert\") pod \"lws-controller-manager-7f7779588f-hkxc7\" (UID: \"a9215463-91d7-47fa-849b-57a626693cb1\") " pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:46.926996 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.926981    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a9215463-91d7-47fa-849b-57a626693cb1-metrics-cert\") pod \"lws-controller-manager-7f7779588f-hkxc7\" (UID: \"a9215463-91d7-47fa-849b-57a626693cb1\") " pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:46.932974 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:46.932946    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlxcv\" (UniqueName: \"kubernetes.io/projected/a9215463-91d7-47fa-849b-57a626693cb1-kube-api-access-jlxcv\") pod \"lws-controller-manager-7f7779588f-hkxc7\" (UID: \"a9215463-91d7-47fa-849b-57a626693cb1\") " pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:47.098475 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:47.098424    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:47.368716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:47.368692    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"]
Sep 08 15:18:47.371616 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:18:47.371586    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9215463_91d7_47fa_849b_57a626693cb1.slice/crio-2c97472b8da05c33137eedbc16df311e03d444291d75f619e6f0510494e60cc2 WatchSource:0}: Error finding container 2c97472b8da05c33137eedbc16df311e03d444291d75f619e6f0510494e60cc2: Status 404 returned error can't find the container with id 2c97472b8da05c33137eedbc16df311e03d444291d75f619e6f0510494e60cc2
Sep 08 15:18:47.522145 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:47.522112    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7" event={"ID":"a9215463-91d7-47fa-849b-57a626693cb1","Type":"ContainerStarted","Data":"2c97472b8da05c33137eedbc16df311e03d444291d75f619e6f0510494e60cc2"}
Sep 08 15:18:48.397413 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:48.397374    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:48.533708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:48.533551    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:50.538650 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:50.538619    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7" event={"ID":"a9215463-91d7-47fa-849b-57a626693cb1","Type":"ContainerStarted","Data":"5a9256774862506bfcb5e7ef04578655a714befab2ae9cca748eb742d307aa66"}
Sep 08 15:18:50.538993 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:50.538738    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:18:50.557963 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:50.557920    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7" podStartSLOduration=2.452220119 podStartE2EDuration="4.557908906s" totalImagesPullingTime="2.105688787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:50.556815573 +0000 UTC m=+578.642765614" watchObservedRunningTime="2026-09-08 15:18:50.557908906 +0000 UTC m=+578.643858947"
Sep 08 15:18:51.483473 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.483439    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:51.543695 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.543665    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"]
Sep 08 15:18:51.555287 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.555265    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"
Sep 08 15:18:51.555875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.555850    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"]
Sep 08 15:18:51.559030 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.559007    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 08 15:18:51.559143 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.559063    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 08 15:18:51.559143 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.559089    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6mm2r\""
Sep 08 15:18:51.559143 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.559135    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 08 15:18:51.559330 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.559138    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 08 15:18:51.564183 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.564160    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:51.662027 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.662009    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2frb\" (UniqueName: \"kubernetes.io/projected/2d1c1fee-f119-466d-96c6-04503d2971c4-kube-api-access-q2frb\") pod \"kube-auth-proxy-5674d6d4b5-jhp44\" (UID: \"2d1c1fee-f119-466d-96c6-04503d2971c4\") " pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"
Sep 08 15:18:51.662124 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.662056    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d1c1fee-f119-466d-96c6-04503d2971c4-tls-certs\") pod \"kube-auth-proxy-5674d6d4b5-jhp44\" (UID: \"2d1c1fee-f119-466d-96c6-04503d2971c4\") " pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"
Sep 08 15:18:51.662124 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.662082    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d1c1fee-f119-466d-96c6-04503d2971c4-tmp\") pod \"kube-auth-proxy-5674d6d4b5-jhp44\" (UID: \"2d1c1fee-f119-466d-96c6-04503d2971c4\") " pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"
Sep 08 15:18:51.762428 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.762377    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2frb\" (UniqueName: \"kubernetes.io/projected/2d1c1fee-f119-466d-96c6-04503d2971c4-kube-api-access-q2frb\") pod \"kube-auth-proxy-5674d6d4b5-jhp44\" (UID: \"2d1c1fee-f119-466d-96c6-04503d2971c4\") " pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"
Sep 08 15:18:51.762428 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.762418    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d1c1fee-f119-466d-96c6-04503d2971c4-tls-certs\") pod \"kube-auth-proxy-5674d6d4b5-jhp44\" (UID: \"2d1c1fee-f119-466d-96c6-04503d2971c4\") " pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"
Sep 08 15:18:51.762554 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.762443    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d1c1fee-f119-466d-96c6-04503d2971c4-tmp\") pod \"kube-auth-proxy-5674d6d4b5-jhp44\" (UID: \"2d1c1fee-f119-466d-96c6-04503d2971c4\") " pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"
Sep 08 15:18:51.764551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.764525    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d1c1fee-f119-466d-96c6-04503d2971c4-tmp\") pod \"kube-auth-proxy-5674d6d4b5-jhp44\" (UID: \"2d1c1fee-f119-466d-96c6-04503d2971c4\") " pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"
Sep 08 15:18:51.764629 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.764574    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d1c1fee-f119-466d-96c6-04503d2971c4-tls-certs\") pod \"kube-auth-proxy-5674d6d4b5-jhp44\" (UID: \"2d1c1fee-f119-466d-96c6-04503d2971c4\") " pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"
Sep 08 15:18:51.771265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.771244    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2frb\" (UniqueName: \"kubernetes.io/projected/2d1c1fee-f119-466d-96c6-04503d2971c4-kube-api-access-q2frb\") pod \"kube-auth-proxy-5674d6d4b5-jhp44\" (UID: \"2d1c1fee-f119-466d-96c6-04503d2971c4\") " pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"
Sep 08 15:18:51.865859 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:51.865842    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"
Sep 08 15:18:52.159293 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:52.159266    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44"]
Sep 08 15:18:52.159798 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:18:52.159764    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d1c1fee_f119_466d_96c6_04503d2971c4.slice/crio-4dcd104fe924a98095debb89a7eb1f152ccc668d09712b47517c200403ff686a WatchSource:0}: Error finding container 4dcd104fe924a98095debb89a7eb1f152ccc668d09712b47517c200403ff686a: Status 404 returned error can't find the container with id 4dcd104fe924a98095debb89a7eb1f152ccc668d09712b47517c200403ff686a
Sep 08 15:18:52.550030 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:52.549970    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44" event={"ID":"2d1c1fee-f119-466d-96c6-04503d2971c4","Type":"ContainerStarted","Data":"4dcd104fe924a98095debb89a7eb1f152ccc668d09712b47517c200403ff686a"}
Sep 08 15:18:52.727649 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:52.727622    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:52.803171 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:52.803091    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:53.778856 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:53.778818    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:56.570069 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:56.570039    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44" event={"ID":"2d1c1fee-f119-466d-96c6-04503d2971c4","Type":"ContainerStarted","Data":"5b9dc7e2920536dd6b22b06cf3201b426b65994f620df789d384068c7ac0b102"}
Sep 08 15:18:56.592194 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:56.592148    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5674d6d4b5-jhp44" podStartSLOduration=1.95361376 podStartE2EDuration="5.592133294s" totalImagesPullingTime="3.638519534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:18:56.590996364 +0000 UTC m=+584.676946407" watchObservedRunningTime="2026-09-08 15:18:56.592133294 +0000 UTC m=+584.678083335"
Sep 08 15:18:57.067290 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:57.067257    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:57.169837 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:57.169817    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:58.198611 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:58.198581    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:58.382677 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:58.382625    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:58.504417 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:58.504349    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:59.584715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:59.584663    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:59.681580 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:59.681547    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:18:59.783703 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:18:59.783674    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:00.912776 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:19:00.912742    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:01.092888 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:19:01.092860    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:01.547258 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:19:01.547237    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7f7779588f-hkxc7"
Sep 08 15:19:02.129474 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:19:02.129434    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:02.203973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:19:02.203947    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:10.898072 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:19:10.898028    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:12.326732 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:19:12.321025    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:19:12.326732 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:19:12.321739    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:19:16.281420 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:19:16.281380    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:19:58.898465 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:19:58.898389    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:00.586963 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:00.586924    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:00.680001 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:00.679974    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:15.255139 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:15.255102    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:15.878017 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:15.877984    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:18.678474 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:18.678447    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:20.472821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:20.472785    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:20.716744 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:20.716715    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:20.869233 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:20.869188    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:21.505950 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:21.505916    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:22.080048 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:22.080018    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:22.984906 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:22.984872    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:23.086347 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:23.086321    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:24.679049 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:24.679013    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:25.079909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:25.079838    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:26.946331 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:26.946297    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:27.058593 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:27.058560    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:28.489940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:28.489905    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:28.593419 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:28.593375    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:32.274470 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:32.274430    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:32.374883 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:32.374845    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:32.478630 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:32.478595    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:32.864961 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:32.864932    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:36.115609 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:36.115575    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:36.250283 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:36.250250    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:36.351711 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:36.351685    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:44.919393 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:44.919359    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:45.021691 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:45.021661    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:47.050719 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:47.050686    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:47.202225 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:47.202192    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:48.464645 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:48.464611    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:48.556694 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:48.556663    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:50.932471 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:50.932440    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:51.061582 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:51.061556    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:56.723551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:56.723514    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:57.860074 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:57.860023    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:20:58.155790 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:20:58.155714    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:04.480284 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:04.480243    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:04.604291 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:04.604263    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:12.828939 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.828909    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"]
Sep 08 15:21:12.831157 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.831140    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:12.838281 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.838261    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 15:21:12.838378 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.838262    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-6cxnb\""
Sep 08 15:21:12.850751 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.850728    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"]
Sep 08 15:21:12.865457 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.865434    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:12.920040 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.920010    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:12.920142 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.920043    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:12.920142 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.920073    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/544ecacb-8bcf-466f-b18e-f21b5509e587-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:12.920263 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.920150    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/544ecacb-8bcf-466f-b18e-f21b5509e587-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:12.920263 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.920186    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:12.920263 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.920256    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:12.920399 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.920297    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tl7t\" (UniqueName: \"kubernetes.io/projected/544ecacb-8bcf-466f-b18e-f21b5509e587-kube-api-access-9tl7t\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:12.920399 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.920324    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:12.920399 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:12.920388    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/544ecacb-8bcf-466f-b18e-f21b5509e587-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.021177 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.021150    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.021255 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.021181    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.021255 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.021214    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/544ecacb-8bcf-466f-b18e-f21b5509e587-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.021255 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.021249    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/544ecacb-8bcf-466f-b18e-f21b5509e587-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.021376 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.021276    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.021376 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.021324    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.021376 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.021361    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tl7t\" (UniqueName: \"kubernetes.io/projected/544ecacb-8bcf-466f-b18e-f21b5509e587-kube-api-access-9tl7t\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.021540 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.021388    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.021540 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.021433    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/544ecacb-8bcf-466f-b18e-f21b5509e587-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.021634 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.021570    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.021695 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.021657    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.021824 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.021806    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.021884 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.021870    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.022015 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.022000    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/544ecacb-8bcf-466f-b18e-f21b5509e587-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.023739 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.023715    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/544ecacb-8bcf-466f-b18e-f21b5509e587-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.023951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.023937    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/544ecacb-8bcf-466f-b18e-f21b5509e587-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.033804 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.033776    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tl7t\" (UniqueName: \"kubernetes.io/projected/544ecacb-8bcf-466f-b18e-f21b5509e587-kube-api-access-9tl7t\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.034478 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.034458    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/544ecacb-8bcf-466f-b18e-f21b5509e587-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qwsx4\" (UID: \"544ecacb-8bcf-466f-b18e-f21b5509e587\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.141177 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.141154    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:13.148539 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.148480    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:13.414886 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:13.414830    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"]
Sep 08 15:21:13.417066 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:21:13.417039    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod544ecacb_8bcf_466f_b18e_f21b5509e587.slice/crio-329280f743067c2fe16c6d1ed9b6b403ed89d24e8244881b8313eb0401021f11 WatchSource:0}: Error finding container 329280f743067c2fe16c6d1ed9b6b403ed89d24e8244881b8313eb0401021f11: Status 404 returned error can't find the container with id 329280f743067c2fe16c6d1ed9b6b403ed89d24e8244881b8313eb0401021f11
Sep 08 15:21:14.154769 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:14.154735    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4" event={"ID":"544ecacb-8bcf-466f-b18e-f21b5509e587","Type":"ContainerStarted","Data":"329280f743067c2fe16c6d1ed9b6b403ed89d24e8244881b8313eb0401021f11"}
Sep 08 15:21:16.275572 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:16.275531    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:21:16.275821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:16.275609    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:21:16.275821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:16.275638    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:21:17.170095 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:17.170055    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4" event={"ID":"544ecacb-8bcf-466f-b18e-f21b5509e587","Type":"ContainerStarted","Data":"68764bbadc0b1fcb14b75e6df4558f8a1ac0bba202cff70648e5378e6e69d5c3"}
Sep 08 15:21:17.237911 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:17.237867    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4" podStartSLOduration=2.381579459 podStartE2EDuration="5.237853043s" totalImagesPullingTime="2.856273584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:21:17.212152664 +0000 UTC m=+725.298102703" watchObservedRunningTime="2026-09-08 15:21:17.237853043 +0000 UTC m=+725.323803083"
Sep 08 15:21:18.142305 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:18.142276    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:18.146305 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:18.146283    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:18.174611 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:18.174588    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:18.175436 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:18.175418    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:21:31.851038 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:31.851000    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:32.277797 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.277769    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:21:32.280134 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.280113    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"
Sep 08 15:21:32.283819 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.283801    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 08 15:21:32.283947 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.283803    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 15:21:32.283947 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.283804    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 15:21:32.284678 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.284662    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vjhlc\""
Sep 08 15:21:32.292124 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.292099    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:21:32.335636 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.335614    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:21:32.365975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.365954    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj5md\" (UniqueName: \"kubernetes.io/projected/1553df11-ed6d-436c-a39f-007c9f4b5b59-kube-api-access-xj5md\") pod \"limitador-limitador-85d9db4d5d-s8s9s\" (UID: \"1553df11-ed6d-436c-a39f-007c9f4b5b59\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"
Sep 08 15:21:32.366040 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.365996    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1553df11-ed6d-436c-a39f-007c9f4b5b59-config-file\") pod \"limitador-limitador-85d9db4d5d-s8s9s\" (UID: \"1553df11-ed6d-436c-a39f-007c9f4b5b59\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"
Sep 08 15:21:32.374640 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.374620    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:32.451759 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.451735    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:32.466821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.466790    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj5md\" (UniqueName: \"kubernetes.io/projected/1553df11-ed6d-436c-a39f-007c9f4b5b59-kube-api-access-xj5md\") pod \"limitador-limitador-85d9db4d5d-s8s9s\" (UID: \"1553df11-ed6d-436c-a39f-007c9f4b5b59\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"
Sep 08 15:21:32.466891 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.466872    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1553df11-ed6d-436c-a39f-007c9f4b5b59-config-file\") pod \"limitador-limitador-85d9db4d5d-s8s9s\" (UID: \"1553df11-ed6d-436c-a39f-007c9f4b5b59\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"
Sep 08 15:21:32.467449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.467432    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1553df11-ed6d-436c-a39f-007c9f4b5b59-config-file\") pod \"limitador-limitador-85d9db4d5d-s8s9s\" (UID: \"1553df11-ed6d-436c-a39f-007c9f4b5b59\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"
Sep 08 15:21:32.476048 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.476025    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj5md\" (UniqueName: \"kubernetes.io/projected/1553df11-ed6d-436c-a39f-007c9f4b5b59-kube-api-access-xj5md\") pod \"limitador-limitador-85d9db4d5d-s8s9s\" (UID: \"1553df11-ed6d-436c-a39f-007c9f4b5b59\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"
Sep 08 15:21:32.591083 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.591036    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"
Sep 08 15:21:32.707534 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.707505    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-fhbdt"]
Sep 08 15:21:32.709825 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.709810    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-fhbdt"
Sep 08 15:21:32.715834 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.715815    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mtxln\""
Sep 08 15:21:32.725000 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.724978    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fhbdt"]
Sep 08 15:21:32.749950 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.749914    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:32.770482 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.770458    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg7qz\" (UniqueName: \"kubernetes.io/projected/75082aa3-7a1a-4df6-ad7b-2f27892f83f3-kube-api-access-pg7qz\") pod \"authorino-b54df94b4-fhbdt\" (UID: \"75082aa3-7a1a-4df6-ad7b-2f27892f83f3\") " pod="kuadrant-system/authorino-b54df94b4-fhbdt"
Sep 08 15:21:32.871365 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.871341    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg7qz\" (UniqueName: \"kubernetes.io/projected/75082aa3-7a1a-4df6-ad7b-2f27892f83f3-kube-api-access-pg7qz\") pod \"authorino-b54df94b4-fhbdt\" (UID: \"75082aa3-7a1a-4df6-ad7b-2f27892f83f3\") " pod="kuadrant-system/authorino-b54df94b4-fhbdt"
Sep 08 15:21:32.894018 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.894001    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:21:32.896415 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:21:32.896395    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1553df11_ed6d_436c_a39f_007c9f4b5b59.slice/crio-95c960403353d8098835b2e6b9aab04ed31db43a1d647925a078074f44621d89 WatchSource:0}: Error finding container 95c960403353d8098835b2e6b9aab04ed31db43a1d647925a078074f44621d89: Status 404 returned error can't find the container with id 95c960403353d8098835b2e6b9aab04ed31db43a1d647925a078074f44621d89
Sep 08 15:21:32.897264 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:32.897244    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg7qz\" (UniqueName: \"kubernetes.io/projected/75082aa3-7a1a-4df6-ad7b-2f27892f83f3-kube-api-access-pg7qz\") pod \"authorino-b54df94b4-fhbdt\" (UID: \"75082aa3-7a1a-4df6-ad7b-2f27892f83f3\") " pod="kuadrant-system/authorino-b54df94b4-fhbdt"
Sep 08 15:21:33.033562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:33.033539    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-fhbdt"
Sep 08 15:21:33.239976 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:33.239907    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s" event={"ID":"1553df11-ed6d-436c-a39f-007c9f4b5b59","Type":"ContainerStarted","Data":"95c960403353d8098835b2e6b9aab04ed31db43a1d647925a078074f44621d89"}
Sep 08 15:21:33.300474 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:33.300446    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fhbdt"]
Sep 08 15:21:33.302582 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:21:33.302550    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75082aa3_7a1a_4df6_ad7b_2f27892f83f3.slice/crio-988f9d6b4988d8c8c8eecda9fb288c7e2fe4cc42619814467bb52d6e8225ed9f WatchSource:0}: Error finding container 988f9d6b4988d8c8c8eecda9fb288c7e2fe4cc42619814467bb52d6e8225ed9f: Status 404 returned error can't find the container with id 988f9d6b4988d8c8c8eecda9fb288c7e2fe4cc42619814467bb52d6e8225ed9f
Sep 08 15:21:33.945610 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:33.945470    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:34.247326 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:34.247238    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-fhbdt" event={"ID":"75082aa3-7a1a-4df6-ad7b-2f27892f83f3","Type":"ContainerStarted","Data":"988f9d6b4988d8c8c8eecda9fb288c7e2fe4cc42619814467bb52d6e8225ed9f"}
Sep 08 15:21:36.258324 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:36.258290    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s" event={"ID":"1553df11-ed6d-436c-a39f-007c9f4b5b59","Type":"ContainerStarted","Data":"8a707ea8e7562017485003cbd9ce532f3cf0057b4b5f0357a20c49f2feb19c45"}
Sep 08 15:21:36.258684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:36.258426    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"
Sep 08 15:21:36.285183 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:36.285135    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s" podStartSLOduration=1.971372645 podStartE2EDuration="4.285115832s" totalImagesPullingTime="2.313743187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:21:36.282184908 +0000 UTC m=+744.368134948" watchObservedRunningTime="2026-09-08 15:21:36.285115832 +0000 UTC m=+744.371065873"
Sep 08 15:21:36.366367 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:36.366339    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fhbdt"]
Sep 08 15:21:37.264632 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:37.264595    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-fhbdt" event={"ID":"75082aa3-7a1a-4df6-ad7b-2f27892f83f3","Type":"ContainerStarted","Data":"392b7d269c587f14e7744ce76ff04d7ae44be6630f05590ee0595a5386f1eee0"}
Sep 08 15:21:37.281202 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:37.281156    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-fhbdt" podStartSLOduration=2.304294135 podStartE2EDuration="5.281141966s" totalImagesPullingTime="2.976847831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:21:37.280697407 +0000 UTC m=+745.366647449" watchObservedRunningTime="2026-09-08 15:21:37.281141966 +0000 UTC m=+745.367092007"
Sep 08 15:21:38.268956 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:38.268918    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-fhbdt" podUID="75082aa3-7a1a-4df6-ad7b-2f27892f83f3" containerName="authorino" containerID="cri-o://392b7d269c587f14e7744ce76ff04d7ae44be6630f05590ee0595a5386f1eee0" gracePeriod=30
Sep 08 15:21:38.509604 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:38.509574    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-fhbdt"
Sep 08 15:21:38.622978 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:38.622958    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pg7qz\" (UniqueName: \"kubernetes.io/projected/75082aa3-7a1a-4df6-ad7b-2f27892f83f3-kube-api-access-pg7qz\") pod \"75082aa3-7a1a-4df6-ad7b-2f27892f83f3\" (UID: \"75082aa3-7a1a-4df6-ad7b-2f27892f83f3\") "
Sep 08 15:21:38.624792 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:38.624772    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75082aa3-7a1a-4df6-ad7b-2f27892f83f3-kube-api-access-pg7qz" (OuterVolumeSpecName: "kube-api-access-pg7qz") pod "75082aa3-7a1a-4df6-ad7b-2f27892f83f3" (UID: "75082aa3-7a1a-4df6-ad7b-2f27892f83f3"). InnerVolumeSpecName "kube-api-access-pg7qz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:21:38.724418 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:38.724395    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg7qz\" (UniqueName: \"kubernetes.io/projected/75082aa3-7a1a-4df6-ad7b-2f27892f83f3-kube-api-access-pg7qz\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:21:39.275244 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:39.275212    2575 generic.go:356] "Generic (PLEG): container finished" podID="75082aa3-7a1a-4df6-ad7b-2f27892f83f3" containerID="392b7d269c587f14e7744ce76ff04d7ae44be6630f05590ee0595a5386f1eee0" exitCode=0
Sep 08 15:21:39.275655 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:39.275255    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-fhbdt" event={"ID":"75082aa3-7a1a-4df6-ad7b-2f27892f83f3","Type":"ContainerDied","Data":"392b7d269c587f14e7744ce76ff04d7ae44be6630f05590ee0595a5386f1eee0"}
Sep 08 15:21:39.275655 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:39.275297    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-fhbdt" event={"ID":"75082aa3-7a1a-4df6-ad7b-2f27892f83f3","Type":"ContainerDied","Data":"988f9d6b4988d8c8c8eecda9fb288c7e2fe4cc42619814467bb52d6e8225ed9f"}
Sep 08 15:21:39.275655 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:39.275314    2575 scope.go:117] "RemoveContainer" containerID="392b7d269c587f14e7744ce76ff04d7ae44be6630f05590ee0595a5386f1eee0"
Sep 08 15:21:39.275655 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:39.275264    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-fhbdt"
Sep 08 15:21:39.284381 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:39.284367    2575 scope.go:117] "RemoveContainer" containerID="392b7d269c587f14e7744ce76ff04d7ae44be6630f05590ee0595a5386f1eee0"
Sep 08 15:21:39.284654 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:21:39.284635    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"392b7d269c587f14e7744ce76ff04d7ae44be6630f05590ee0595a5386f1eee0\": container with ID starting with 392b7d269c587f14e7744ce76ff04d7ae44be6630f05590ee0595a5386f1eee0 not found: ID does not exist" containerID="392b7d269c587f14e7744ce76ff04d7ae44be6630f05590ee0595a5386f1eee0"
Sep 08 15:21:39.284721 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:39.284661    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"392b7d269c587f14e7744ce76ff04d7ae44be6630f05590ee0595a5386f1eee0"} err="failed to get container status \"392b7d269c587f14e7744ce76ff04d7ae44be6630f05590ee0595a5386f1eee0\": rpc error: code = NotFound desc = could not find container \"392b7d269c587f14e7744ce76ff04d7ae44be6630f05590ee0595a5386f1eee0\": container with ID starting with 392b7d269c587f14e7744ce76ff04d7ae44be6630f05590ee0595a5386f1eee0 not found: ID does not exist"
Sep 08 15:21:39.297118 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:39.297097    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fhbdt"]
Sep 08 15:21:39.301020 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:39.300996    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fhbdt"]
Sep 08 15:21:40.430745 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:40.430714    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75082aa3-7a1a-4df6-ad7b-2f27892f83f3" path="/var/lib/kubelet/pods/75082aa3-7a1a-4df6-ad7b-2f27892f83f3/volumes"
Sep 08 15:21:47.265777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:47.265746    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"
Sep 08 15:21:49.855671 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:49.855635    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:49.954157 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:49.954127    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:21:50.423423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:21:50.423390    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:03.858324 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:03.858256    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:04.074563 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.074539    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-f8gj6"]
Sep 08 15:22:04.074865 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.074853    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75082aa3-7a1a-4df6-ad7b-2f27892f83f3" containerName="authorino"
Sep 08 15:22:04.074915 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.074866    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="75082aa3-7a1a-4df6-ad7b-2f27892f83f3" containerName="authorino"
Sep 08 15:22:04.074950 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.074932    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="75082aa3-7a1a-4df6-ad7b-2f27892f83f3" containerName="authorino"
Sep 08 15:22:04.082918 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.082899    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-f8gj6"
Sep 08 15:22:04.085971 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.085951    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-f8gj6"]
Sep 08 15:22:04.086279 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.086264    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mtxln\""
Sep 08 15:22:04.217396 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.217346    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v78qb\" (UniqueName: \"kubernetes.io/projected/ba667a05-aafa-4cf5-af6a-a07a6e0666f0-kube-api-access-v78qb\") pod \"authorino-5dc7ccb5f-f8gj6\" (UID: \"ba667a05-aafa-4cf5-af6a-a07a6e0666f0\") " pod="kuadrant-system/authorino-5dc7ccb5f-f8gj6"
Sep 08 15:22:04.300146 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.300119    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:04.318302 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.318283    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v78qb\" (UniqueName: \"kubernetes.io/projected/ba667a05-aafa-4cf5-af6a-a07a6e0666f0-kube-api-access-v78qb\") pod \"authorino-5dc7ccb5f-f8gj6\" (UID: \"ba667a05-aafa-4cf5-af6a-a07a6e0666f0\") " pod="kuadrant-system/authorino-5dc7ccb5f-f8gj6"
Sep 08 15:22:04.329979 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.329961    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v78qb\" (UniqueName: \"kubernetes.io/projected/ba667a05-aafa-4cf5-af6a-a07a6e0666f0-kube-api-access-v78qb\") pod \"authorino-5dc7ccb5f-f8gj6\" (UID: \"ba667a05-aafa-4cf5-af6a-a07a6e0666f0\") " pod="kuadrant-system/authorino-5dc7ccb5f-f8gj6"
Sep 08 15:22:04.339921 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.339174    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-f8gj6"]
Sep 08 15:22:04.339921 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.339736    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-f8gj6"
Sep 08 15:22:04.370352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.370328    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84f479bd4f-lrr2t"]
Sep 08 15:22:04.373919 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.373899    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84f479bd4f-lrr2t"
Sep 08 15:22:04.382677 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.382655    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84f479bd4f-lrr2t"]
Sep 08 15:22:04.401140 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.401114    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84f479bd4f-lrr2t"]
Sep 08 15:22:04.401667 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:22:04.401640    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-mr55m], unattached volumes=[], failed to process volumes=[kube-api-access-mr55m]: context canceled" pod="kuadrant-system/authorino-84f479bd4f-lrr2t" podUID="398e5519-7c6c-4cde-a7af-6c82644651f8"
Sep 08 15:22:04.432663 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.432634    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65f6d49b67-mdf7v"]
Sep 08 15:22:04.435984 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.435965    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65f6d49b67-mdf7v"
Sep 08 15:22:04.438579 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.438564    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 08 15:22:04.444306 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.444283    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65f6d49b67-mdf7v"]
Sep 08 15:22:04.519895 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.519850    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk29w\" (UniqueName: \"kubernetes.io/projected/f16c0c75-f55b-4270-8835-fb71c63d5ad0-kube-api-access-vk29w\") pod \"authorino-65f6d49b67-mdf7v\" (UID: \"f16c0c75-f55b-4270-8835-fb71c63d5ad0\") " pod="kuadrant-system/authorino-65f6d49b67-mdf7v"
Sep 08 15:22:04.519980 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.519909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr55m\" (UniqueName: \"kubernetes.io/projected/398e5519-7c6c-4cde-a7af-6c82644651f8-kube-api-access-mr55m\") pod \"authorino-84f479bd4f-lrr2t\" (UID: \"398e5519-7c6c-4cde-a7af-6c82644651f8\") " pod="kuadrant-system/authorino-84f479bd4f-lrr2t"
Sep 08 15:22:04.520018 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.519986    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f16c0c75-f55b-4270-8835-fb71c63d5ad0-tls-cert\") pod \"authorino-65f6d49b67-mdf7v\" (UID: \"f16c0c75-f55b-4270-8835-fb71c63d5ad0\") " pod="kuadrant-system/authorino-65f6d49b67-mdf7v"
Sep 08 15:22:04.611248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.611227    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-f8gj6"]
Sep 08 15:22:04.613475 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:22:04.613452    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba667a05_aafa_4cf5_af6a_a07a6e0666f0.slice/crio-ea519d32736cdcff833cd55361bb59c801a2b1f9d32e449d0fe8b9f810c9a7ba WatchSource:0}: Error finding container ea519d32736cdcff833cd55361bb59c801a2b1f9d32e449d0fe8b9f810c9a7ba: Status 404 returned error can't find the container with id ea519d32736cdcff833cd55361bb59c801a2b1f9d32e449d0fe8b9f810c9a7ba
Sep 08 15:22:04.620387 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.620363    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f16c0c75-f55b-4270-8835-fb71c63d5ad0-tls-cert\") pod \"authorino-65f6d49b67-mdf7v\" (UID: \"f16c0c75-f55b-4270-8835-fb71c63d5ad0\") " pod="kuadrant-system/authorino-65f6d49b67-mdf7v"
Sep 08 15:22:04.620480 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.620429    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk29w\" (UniqueName: \"kubernetes.io/projected/f16c0c75-f55b-4270-8835-fb71c63d5ad0-kube-api-access-vk29w\") pod \"authorino-65f6d49b67-mdf7v\" (UID: \"f16c0c75-f55b-4270-8835-fb71c63d5ad0\") " pod="kuadrant-system/authorino-65f6d49b67-mdf7v"
Sep 08 15:22:04.620565 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.620517    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr55m\" (UniqueName: \"kubernetes.io/projected/398e5519-7c6c-4cde-a7af-6c82644651f8-kube-api-access-mr55m\") pod \"authorino-84f479bd4f-lrr2t\" (UID: \"398e5519-7c6c-4cde-a7af-6c82644651f8\") " pod="kuadrant-system/authorino-84f479bd4f-lrr2t"
Sep 08 15:22:04.622677 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.622657    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f16c0c75-f55b-4270-8835-fb71c63d5ad0-tls-cert\") pod \"authorino-65f6d49b67-mdf7v\" (UID: \"f16c0c75-f55b-4270-8835-fb71c63d5ad0\") " pod="kuadrant-system/authorino-65f6d49b67-mdf7v"
Sep 08 15:22:04.629325 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.629302    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr55m\" (UniqueName: \"kubernetes.io/projected/398e5519-7c6c-4cde-a7af-6c82644651f8-kube-api-access-mr55m\") pod \"authorino-84f479bd4f-lrr2t\" (UID: \"398e5519-7c6c-4cde-a7af-6c82644651f8\") " pod="kuadrant-system/authorino-84f479bd4f-lrr2t"
Sep 08 15:22:04.629443 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.629425    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk29w\" (UniqueName: \"kubernetes.io/projected/f16c0c75-f55b-4270-8835-fb71c63d5ad0-kube-api-access-vk29w\") pod \"authorino-65f6d49b67-mdf7v\" (UID: \"f16c0c75-f55b-4270-8835-fb71c63d5ad0\") " pod="kuadrant-system/authorino-65f6d49b67-mdf7v"
Sep 08 15:22:04.756951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:04.756927    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65f6d49b67-mdf7v"
Sep 08 15:22:05.027180 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:05.027136    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65f6d49b67-mdf7v"]
Sep 08 15:22:05.029297 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:22:05.029269    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf16c0c75_f55b_4270_8835_fb71c63d5ad0.slice/crio-82995f41a57adb5c3aaf6554e6c904111a956c8e555ccc40e9ee0a72a4a7aea8 WatchSource:0}: Error finding container 82995f41a57adb5c3aaf6554e6c904111a956c8e555ccc40e9ee0a72a4a7aea8: Status 404 returned error can't find the container with id 82995f41a57adb5c3aaf6554e6c904111a956c8e555ccc40e9ee0a72a4a7aea8
Sep 08 15:22:05.392416 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:05.392390    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-f8gj6" event={"ID":"ba667a05-aafa-4cf5-af6a-a07a6e0666f0","Type":"ContainerStarted","Data":"ea519d32736cdcff833cd55361bb59c801a2b1f9d32e449d0fe8b9f810c9a7ba"}
Sep 08 15:22:05.393482 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:05.393463    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84f479bd4f-lrr2t"
Sep 08 15:22:05.393602 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:05.393506    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65f6d49b67-mdf7v" event={"ID":"f16c0c75-f55b-4270-8835-fb71c63d5ad0","Type":"ContainerStarted","Data":"82995f41a57adb5c3aaf6554e6c904111a956c8e555ccc40e9ee0a72a4a7aea8"}
Sep 08 15:22:05.403930 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:05.403905    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84f479bd4f-lrr2t"
Sep 08 15:22:05.527559 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:05.527537    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr55m\" (UniqueName: \"kubernetes.io/projected/398e5519-7c6c-4cde-a7af-6c82644651f8-kube-api-access-mr55m\") pod \"398e5519-7c6c-4cde-a7af-6c82644651f8\" (UID: \"398e5519-7c6c-4cde-a7af-6c82644651f8\") "
Sep 08 15:22:05.529397 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:05.529374    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/398e5519-7c6c-4cde-a7af-6c82644651f8-kube-api-access-mr55m" (OuterVolumeSpecName: "kube-api-access-mr55m") pod "398e5519-7c6c-4cde-a7af-6c82644651f8" (UID: "398e5519-7c6c-4cde-a7af-6c82644651f8"). InnerVolumeSpecName "kube-api-access-mr55m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:22:05.628791 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:05.628770    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr55m\" (UniqueName: \"kubernetes.io/projected/398e5519-7c6c-4cde-a7af-6c82644651f8-kube-api-access-mr55m\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:22:06.401704 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:06.401663    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65f6d49b67-mdf7v" event={"ID":"f16c0c75-f55b-4270-8835-fb71c63d5ad0","Type":"ContainerStarted","Data":"0aa36dd21bceaef3b6127083b9d5d255cc6d0bf53a0c2713c7a423af88b156c6"}
Sep 08 15:22:06.403255 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:06.403228    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-f8gj6" event={"ID":"ba667a05-aafa-4cf5-af6a-a07a6e0666f0","Type":"ContainerStarted","Data":"e136e5000a081f896f0a890542cc6d56ef20e8129b169a6705facff3e939d610"}
Sep 08 15:22:06.403392 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:06.403234    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84f479bd4f-lrr2t"
Sep 08 15:22:06.403392 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:06.403299    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5dc7ccb5f-f8gj6" podUID="ba667a05-aafa-4cf5-af6a-a07a6e0666f0" containerName="authorino" containerID="cri-o://e136e5000a081f896f0a890542cc6d56ef20e8129b169a6705facff3e939d610" gracePeriod=30
Sep 08 15:22:06.420935 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:06.420895    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-65f6d49b67-mdf7v" podStartSLOduration=1.773934941 podStartE2EDuration="2.420883084s" totalImagesPullingTime="646.948143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:22:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:22:06.419674568 +0000 UTC m=+774.505624613" watchObservedRunningTime="2026-09-08 15:22:06.420883084 +0000 UTC m=+774.506833125"
Sep 08 15:22:06.437027 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:06.436990    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5dc7ccb5f-f8gj6" podStartSLOduration=1.616146605 podStartE2EDuration="2.436977427s" totalImagesPullingTime="820.830822ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:22:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:22:06.436078704 +0000 UTC m=+774.522028758" watchObservedRunningTime="2026-09-08 15:22:06.436977427 +0000 UTC m=+774.522927467"
Sep 08 15:22:06.464598 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:06.464572    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84f479bd4f-lrr2t"]
Sep 08 15:22:06.472298 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:06.472274    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-84f479bd4f-lrr2t"]
Sep 08 15:22:06.640148 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:06.640126    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-f8gj6"
Sep 08 15:22:06.737162 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:06.737092    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v78qb\" (UniqueName: \"kubernetes.io/projected/ba667a05-aafa-4cf5-af6a-a07a6e0666f0-kube-api-access-v78qb\") pod \"ba667a05-aafa-4cf5-af6a-a07a6e0666f0\" (UID: \"ba667a05-aafa-4cf5-af6a-a07a6e0666f0\") "
Sep 08 15:22:06.738919 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:06.738898    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba667a05-aafa-4cf5-af6a-a07a6e0666f0-kube-api-access-v78qb" (OuterVolumeSpecName: "kube-api-access-v78qb") pod "ba667a05-aafa-4cf5-af6a-a07a6e0666f0" (UID: "ba667a05-aafa-4cf5-af6a-a07a6e0666f0"). InnerVolumeSpecName "kube-api-access-v78qb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:22:06.837720 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:06.837696    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v78qb\" (UniqueName: \"kubernetes.io/projected/ba667a05-aafa-4cf5-af6a-a07a6e0666f0-kube-api-access-v78qb\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:22:07.409647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:07.409618    2575 generic.go:356] "Generic (PLEG): container finished" podID="ba667a05-aafa-4cf5-af6a-a07a6e0666f0" containerID="e136e5000a081f896f0a890542cc6d56ef20e8129b169a6705facff3e939d610" exitCode=0
Sep 08 15:22:07.410039 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:07.409665    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-f8gj6"
Sep 08 15:22:07.410039 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:07.409705    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-f8gj6" event={"ID":"ba667a05-aafa-4cf5-af6a-a07a6e0666f0","Type":"ContainerDied","Data":"e136e5000a081f896f0a890542cc6d56ef20e8129b169a6705facff3e939d610"}
Sep 08 15:22:07.410039 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:07.409752    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc7ccb5f-f8gj6" event={"ID":"ba667a05-aafa-4cf5-af6a-a07a6e0666f0","Type":"ContainerDied","Data":"ea519d32736cdcff833cd55361bb59c801a2b1f9d32e449d0fe8b9f810c9a7ba"}
Sep 08 15:22:07.410039 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:07.409779    2575 scope.go:117] "RemoveContainer" containerID="e136e5000a081f896f0a890542cc6d56ef20e8129b169a6705facff3e939d610"
Sep 08 15:22:07.417997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:07.417978    2575 scope.go:117] "RemoveContainer" containerID="e136e5000a081f896f0a890542cc6d56ef20e8129b169a6705facff3e939d610"
Sep 08 15:22:07.418220 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:22:07.418204    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e136e5000a081f896f0a890542cc6d56ef20e8129b169a6705facff3e939d610\": container with ID starting with e136e5000a081f896f0a890542cc6d56ef20e8129b169a6705facff3e939d610 not found: ID does not exist" containerID="e136e5000a081f896f0a890542cc6d56ef20e8129b169a6705facff3e939d610"
Sep 08 15:22:07.418282 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:07.418231    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e136e5000a081f896f0a890542cc6d56ef20e8129b169a6705facff3e939d610"} err="failed to get container status \"e136e5000a081f896f0a890542cc6d56ef20e8129b169a6705facff3e939d610\": rpc error: code = NotFound desc = could not find container \"e136e5000a081f896f0a890542cc6d56ef20e8129b169a6705facff3e939d610\": container with ID starting with e136e5000a081f896f0a890542cc6d56ef20e8129b169a6705facff3e939d610 not found: ID does not exist"
Sep 08 15:22:07.432119 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:07.432092    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-f8gj6"]
Sep 08 15:22:07.434827 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:07.434807    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-f8gj6"]
Sep 08 15:22:08.431409 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:08.431365    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="398e5519-7c6c-4cde-a7af-6c82644651f8" path="/var/lib/kubelet/pods/398e5519-7c6c-4cde-a7af-6c82644651f8/volumes"
Sep 08 15:22:08.431887 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:08.431764    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba667a05-aafa-4cf5-af6a-a07a6e0666f0" path="/var/lib/kubelet/pods/ba667a05-aafa-4cf5-af6a-a07a6e0666f0/volumes"
Sep 08 15:22:11.401485 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:11.401456    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:11.713975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:11.713898    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:11.819404 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:11.819364    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:14.927248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:14.927194    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:15.025878 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:15.025845    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:47.599282 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:47.599191    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:47.779420 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:47.779384    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:49.051094 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:49.051060    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:49.183650 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:49.183613    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:49.417120 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:49.417091    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:49.819709 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:49.819638    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:49.921867 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:49.921834    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:50.036206 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:50.036172    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:50.062562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:50.062485    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:22:50.170939 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:50.170874    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:22:59.645552 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:59.645476    2575 generic.go:356] "Generic (PLEG): container finished" podID="544ecacb-8bcf-466f-b18e-f21b5509e587" containerID="68764bbadc0b1fcb14b75e6df4558f8a1ac0bba202cff70648e5378e6e69d5c3" exitCode=0
Sep 08 15:22:59.645552 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:59.645549    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4" event={"ID":"544ecacb-8bcf-466f-b18e-f21b5509e587","Type":"ContainerDied","Data":"68764bbadc0b1fcb14b75e6df4558f8a1ac0bba202cff70648e5378e6e69d5c3"}
Sep 08 15:22:59.646006 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:59.645990    2575 scope.go:117] "RemoveContainer" containerID="68764bbadc0b1fcb14b75e6df4558f8a1ac0bba202cff70648e5378e6e69d5c3"
Sep 08 15:22:59.646988 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:59.646957    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:22:59.647081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:59.647040    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:22:59.647143 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:22:59.647084    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:23:00.653136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:00.653105    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4" event={"ID":"544ecacb-8bcf-466f-b18e-f21b5509e587","Type":"ContainerStarted","Data":"dbb3c87ab4e5c0816d7da3d5924218d8b2254327e678a1b5e4a42f825737980c"}
Sep 08 15:23:00.890993 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:00.890950    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:01.008412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:01.008308    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:01.141592 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:01.141553    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:23:01.142795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:01.142773    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qwsx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 08 15:23:01.142903 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:01.142820    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4" podUID="544ecacb-8bcf-466f-b18e-f21b5509e587" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 08 15:23:02.142015 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:02.141978    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qwsx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 08 15:23:02.142354 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:02.142035    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4" podUID="544ecacb-8bcf-466f-b18e-f21b5509e587" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 08 15:23:03.142280 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:03.142245    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qwsx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 08 15:23:03.142668 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:03.142299    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4" podUID="544ecacb-8bcf-466f-b18e-f21b5509e587" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 08 15:23:03.175094 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:03.175070    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:23:04.141863 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:04.141826    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qwsx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 08 15:23:04.142023 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:04.141881    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4" podUID="544ecacb-8bcf-466f-b18e-f21b5509e587" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 08 15:23:05.141990 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:05.141955    2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qwsx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 08 15:23:05.142358 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:05.142024    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4" podUID="544ecacb-8bcf-466f-b18e-f21b5509e587" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 08 15:23:06.145346 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:06.145313    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:23:06.146186 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:06.146167    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qwsx4"
Sep 08 15:23:09.562394 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:09.562353    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:09.643473 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:09.643441    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:15.078104 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:15.078062    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:15.538853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:15.538816    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:23:28.449991 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.449959    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"]
Sep 08 15:23:28.450436 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.450308    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba667a05-aafa-4cf5-af6a-a07a6e0666f0" containerName="authorino"
Sep 08 15:23:28.450436 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.450319    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba667a05-aafa-4cf5-af6a-a07a6e0666f0" containerName="authorino"
Sep 08 15:23:28.450436 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.450393    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba667a05-aafa-4cf5-af6a-a07a6e0666f0" containerName="authorino"
Sep 08 15:23:28.453717 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.453700    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.457370 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.457351    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 08 15:23:28.457370 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.457363    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p82cs\""
Sep 08 15:23:28.457558 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.457394    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 08 15:23:28.457558 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.457353    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 08 15:23:28.464527 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.464506    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"]
Sep 08 15:23:28.535541 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.535511    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/986d3075-ed53-4af2-a506-91e614440814-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.535715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.535547    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/986d3075-ed53-4af2-a506-91e614440814-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.535715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.535607    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/986d3075-ed53-4af2-a506-91e614440814-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.535715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.535639    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/986d3075-ed53-4af2-a506-91e614440814-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.535715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.535711    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94c6w\" (UniqueName: \"kubernetes.io/projected/986d3075-ed53-4af2-a506-91e614440814-kube-api-access-94c6w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.535925 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.535762    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/986d3075-ed53-4af2-a506-91e614440814-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.547027 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.546998    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:28.636554 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.636527    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/986d3075-ed53-4af2-a506-91e614440814-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.636683 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.636565    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/986d3075-ed53-4af2-a506-91e614440814-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.636683 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.636594    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94c6w\" (UniqueName: \"kubernetes.io/projected/986d3075-ed53-4af2-a506-91e614440814-kube-api-access-94c6w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.636683 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.636624    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/986d3075-ed53-4af2-a506-91e614440814-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.636820 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.636719    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/986d3075-ed53-4af2-a506-91e614440814-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.636820 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.636757    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/986d3075-ed53-4af2-a506-91e614440814-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.636916 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.636905    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/986d3075-ed53-4af2-a506-91e614440814-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.636991 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.636969    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/986d3075-ed53-4af2-a506-91e614440814-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.637097 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.637079    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/986d3075-ed53-4af2-a506-91e614440814-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.638747 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.638727    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/986d3075-ed53-4af2-a506-91e614440814-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.638982 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.638964    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/986d3075-ed53-4af2-a506-91e614440814-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.645185 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.645163    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94c6w\" (UniqueName: \"kubernetes.io/projected/986d3075-ed53-4af2-a506-91e614440814-kube-api-access-94c6w\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ndt97\" (UID: \"986d3075-ed53-4af2-a506-91e614440814\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:28.764766 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:28.764687    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:29.037302 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:29.037276    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"]
Sep 08 15:23:29.039881 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:23:29.039839    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod986d3075_ed53_4af2_a506_91e614440814.slice/crio-62576387e95f44c07e5a348f981d1e421243ef32230b86f868e4f18d23d504e3 WatchSource:0}: Error finding container 62576387e95f44c07e5a348f981d1e421243ef32230b86f868e4f18d23d504e3: Status 404 returned error can't find the container with id 62576387e95f44c07e5a348f981d1e421243ef32230b86f868e4f18d23d504e3
Sep 08 15:23:29.241020 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:29.240976    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:23:29.786874 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:29.786829    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97" event={"ID":"986d3075-ed53-4af2-a506-91e614440814","Type":"ContainerStarted","Data":"62576387e95f44c07e5a348f981d1e421243ef32230b86f868e4f18d23d504e3"}
Sep 08 15:23:30.792128 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:30.792092    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97" event={"ID":"986d3075-ed53-4af2-a506-91e614440814","Type":"ContainerStarted","Data":"3badaf012014736c4704bc06ca5da24fd27e94cffa0fb883f98c39247b17a0fd"}
Sep 08 15:23:30.813632 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:30.813581    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97" podStartSLOduration=1.735161405 podStartE2EDuration="2.813564512s" totalImagesPullingTime="1.078403107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:23:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:23:30.811901567 +0000 UTC m=+858.897851609" watchObservedRunningTime="2026-09-08 15:23:30.813564512 +0000 UTC m=+858.899514554"
Sep 08 15:23:31.519461 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:31.519421    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:38.193218 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:38.193152    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:38.765766 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:38.765731    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:38.765954 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:38.765794    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:38.778267 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:38.778241    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:38.842133 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:38.842106    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ndt97"
Sep 08 15:23:39.142877 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:39.142843    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:23:45.971158 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:45.971109    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:23:47.264315 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:23:47.264283    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:24:02.207182 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:02.207131    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:03.653718 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:03.653670    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:24:06.839317 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.839233    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"]
Sep 08 15:24:06.843602 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.843580    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.846407 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.846386    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 08 15:24:06.855637 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.855615    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"]
Sep 08 15:24:06.860511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.860471    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl5mv\" (UniqueName: \"kubernetes.io/projected/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-kube-api-access-tl5mv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.860617 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.860531    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.860617 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.860551    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.860617 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.860592    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.860617 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.860612    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.860933 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.860652    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.943473 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.943444    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:06.961826 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.961804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl5mv\" (UniqueName: \"kubernetes.io/projected/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-kube-api-access-tl5mv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.961956 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.961840    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.961956 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.961858    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.961956 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.961892    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.961956 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.961916    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.962152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.961970    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.962272 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.962248    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.962335 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.962282    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.962374 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.962331    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.963958 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.963938    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.964277 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.964260    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:06.970449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:06.970425    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl5mv\" (UniqueName: \"kubernetes.io/projected/6dfd08ce-a73f-4134-b3b7-dc18e5a8f878-kube-api-access-tl5mv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-glqq4\" (UID: \"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:07.179534 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:07.179482    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:07.460168 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:07.460142    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"]
Sep 08 15:24:07.461951 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:24:07.461927    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dfd08ce_a73f_4134_b3b7_dc18e5a8f878.slice/crio-bd6841ec2ff1e01f714ccf641fafe98debe9bc80d0fc3be312d28f1c806fba02 WatchSource:0}: Error finding container bd6841ec2ff1e01f714ccf641fafe98debe9bc80d0fc3be312d28f1c806fba02: Status 404 returned error can't find the container with id bd6841ec2ff1e01f714ccf641fafe98debe9bc80d0fc3be312d28f1c806fba02
Sep 08 15:24:07.463232 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:07.463211    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:24:07.967410 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:07.967375    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4" event={"ID":"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878","Type":"ContainerStarted","Data":"7bbf2f2249942935a3070bdc3cfa169fe0c97f3a2d20eb662eca9080bdd0ea36"}
Sep 08 15:24:07.967781 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:07.967433    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4" event={"ID":"6dfd08ce-a73f-4134-b3b7-dc18e5a8f878","Type":"ContainerStarted","Data":"bd6841ec2ff1e01f714ccf641fafe98debe9bc80d0fc3be312d28f1c806fba02"}
Sep 08 15:24:07.989228 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:07.989132    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4" podStartSLOduration=1.762941868 podStartE2EDuration="1.989114908s" totalImagesPullingTime="226.17304ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:24:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:24:07.987338885 +0000 UTC m=+896.073288928" watchObservedRunningTime="2026-09-08 15:24:07.989114908 +0000 UTC m=+896.075064951"
Sep 08 15:24:08.440179 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:08.440145    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:24:12.355782 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:12.355751    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:24:12.358889 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:12.358867    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:24:17.179915 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:17.179872    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:17.180287 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:17.179943    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:17.192647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:17.192620    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:18.028402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:18.028376    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-glqq4"
Sep 08 15:24:20.527505 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:20.527457    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:22.447302 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:22.447269    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:26.456154 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.456115    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"]
Sep 08 15:24:26.460273 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.460254    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.463265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.463245    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 08 15:24:26.472324 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.472288    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"]
Sep 08 15:24:26.505447 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.505420    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:24:26.535943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.535921    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.536066 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.535949    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.536066 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.535978    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.536066 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.536042    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9qvb\" (UniqueName: \"kubernetes.io/projected/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-kube-api-access-j9qvb\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.536208 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.536123    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.536208 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.536194    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.637458 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.637414    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.637649 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.637470    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.637649 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.637553    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.637649 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.637580    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.637649 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.637621    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.637856 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.637655    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9qvb\" (UniqueName: \"kubernetes.io/projected/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-kube-api-access-j9qvb\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.638012 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.637980    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.638012 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.638002    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.638200 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.638087    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.639752 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.639727    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.639911 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.639895    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.651693 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.651674    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9qvb\" (UniqueName: \"kubernetes.io/projected/a8c8d6a0-cae9-4f40-8159-bf2d571d1df9-kube-api-access-j9qvb\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ndq46\" (UID: \"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:26.769232 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:26.769145    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:27.042594 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:27.042570    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"]
Sep 08 15:24:27.044704 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:24:27.044673    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8c8d6a0_cae9_4f40_8159_bf2d571d1df9.slice/crio-83f475c4ce12881cfbe7b574d2c1b0795008143eb6a5a8850ea7c9015aed426c WatchSource:0}: Error finding container 83f475c4ce12881cfbe7b574d2c1b0795008143eb6a5a8850ea7c9015aed426c: Status 404 returned error can't find the container with id 83f475c4ce12881cfbe7b574d2c1b0795008143eb6a5a8850ea7c9015aed426c
Sep 08 15:24:27.058792 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:27.058764    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46" event={"ID":"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9","Type":"ContainerStarted","Data":"83f475c4ce12881cfbe7b574d2c1b0795008143eb6a5a8850ea7c9015aed426c"}
Sep 08 15:24:28.065051 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:28.065011    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46" event={"ID":"a8c8d6a0-cae9-4f40-8159-bf2d571d1df9","Type":"ContainerStarted","Data":"7724b72befb45dedf0e5adfd9f7a4a4efff86f9e27922425e87355acd5d1edc0"}
Sep 08 15:24:28.086345 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:28.086297    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46" podStartSLOduration=1.882644403 podStartE2EDuration="2.08628133s" totalImagesPullingTime="203.636927ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:24:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:24:28.084529431 +0000 UTC m=+916.170479474" watchObservedRunningTime="2026-09-08 15:24:28.08628133 +0000 UTC m=+916.172231373"
Sep 08 15:24:28.649332 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:28.649290    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:24:36.770324 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:36.770285    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:36.770757 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:36.770623    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:36.782902 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:36.782880    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:24:37.122178 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:24:37.122147    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ndq46"
Sep 08 15:25:37.854189 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:37.854108    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65f6d49b67-mdf7v"]
Sep 08 15:25:37.854713 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:37.854335    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-65f6d49b67-mdf7v" podUID="f16c0c75-f55b-4270-8835-fb71c63d5ad0" containerName="authorino" containerID="cri-o://0aa36dd21bceaef3b6127083b9d5d255cc6d0bf53a0c2713c7a423af88b156c6" gracePeriod=30
Sep 08 15:25:38.103706 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.103682    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65f6d49b67-mdf7v"
Sep 08 15:25:38.220996 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.220917    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f16c0c75-f55b-4270-8835-fb71c63d5ad0-tls-cert\") pod \"f16c0c75-f55b-4270-8835-fb71c63d5ad0\" (UID: \"f16c0c75-f55b-4270-8835-fb71c63d5ad0\") "
Sep 08 15:25:38.221139 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.221032    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vk29w\" (UniqueName: \"kubernetes.io/projected/f16c0c75-f55b-4270-8835-fb71c63d5ad0-kube-api-access-vk29w\") pod \"f16c0c75-f55b-4270-8835-fb71c63d5ad0\" (UID: \"f16c0c75-f55b-4270-8835-fb71c63d5ad0\") "
Sep 08 15:25:38.222971 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.222945    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f16c0c75-f55b-4270-8835-fb71c63d5ad0-kube-api-access-vk29w" (OuterVolumeSpecName: "kube-api-access-vk29w") pod "f16c0c75-f55b-4270-8835-fb71c63d5ad0" (UID: "f16c0c75-f55b-4270-8835-fb71c63d5ad0"). InnerVolumeSpecName "kube-api-access-vk29w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:25:38.230941 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.230915    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16c0c75-f55b-4270-8835-fb71c63d5ad0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f16c0c75-f55b-4270-8835-fb71c63d5ad0" (UID: "f16c0c75-f55b-4270-8835-fb71c63d5ad0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:25:38.321781 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.321743    2575 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f16c0c75-f55b-4270-8835-fb71c63d5ad0-tls-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:25:38.321781 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.321777    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk29w\" (UniqueName: \"kubernetes.io/projected/f16c0c75-f55b-4270-8835-fb71c63d5ad0-kube-api-access-vk29w\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:25:38.397743 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.397711    2575 generic.go:356] "Generic (PLEG): container finished" podID="f16c0c75-f55b-4270-8835-fb71c63d5ad0" containerID="0aa36dd21bceaef3b6127083b9d5d255cc6d0bf53a0c2713c7a423af88b156c6" exitCode=0
Sep 08 15:25:38.397901 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.397746    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65f6d49b67-mdf7v"
Sep 08 15:25:38.397901 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.397766    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65f6d49b67-mdf7v" event={"ID":"f16c0c75-f55b-4270-8835-fb71c63d5ad0","Type":"ContainerDied","Data":"0aa36dd21bceaef3b6127083b9d5d255cc6d0bf53a0c2713c7a423af88b156c6"}
Sep 08 15:25:38.397901 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.397807    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65f6d49b67-mdf7v" event={"ID":"f16c0c75-f55b-4270-8835-fb71c63d5ad0","Type":"ContainerDied","Data":"82995f41a57adb5c3aaf6554e6c904111a956c8e555ccc40e9ee0a72a4a7aea8"}
Sep 08 15:25:38.397901 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.397830    2575 scope.go:117] "RemoveContainer" containerID="0aa36dd21bceaef3b6127083b9d5d255cc6d0bf53a0c2713c7a423af88b156c6"
Sep 08 15:25:38.406878 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.406851    2575 scope.go:117] "RemoveContainer" containerID="0aa36dd21bceaef3b6127083b9d5d255cc6d0bf53a0c2713c7a423af88b156c6"
Sep 08 15:25:38.407124 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:25:38.407106    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0aa36dd21bceaef3b6127083b9d5d255cc6d0bf53a0c2713c7a423af88b156c6\": container with ID starting with 0aa36dd21bceaef3b6127083b9d5d255cc6d0bf53a0c2713c7a423af88b156c6 not found: ID does not exist" containerID="0aa36dd21bceaef3b6127083b9d5d255cc6d0bf53a0c2713c7a423af88b156c6"
Sep 08 15:25:38.407166 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.407132    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0aa36dd21bceaef3b6127083b9d5d255cc6d0bf53a0c2713c7a423af88b156c6"} err="failed to get container status \"0aa36dd21bceaef3b6127083b9d5d255cc6d0bf53a0c2713c7a423af88b156c6\": rpc error: code = NotFound desc = could not find container \"0aa36dd21bceaef3b6127083b9d5d255cc6d0bf53a0c2713c7a423af88b156c6\": container with ID starting with 0aa36dd21bceaef3b6127083b9d5d255cc6d0bf53a0c2713c7a423af88b156c6 not found: ID does not exist"
Sep 08 15:25:38.425973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.425952    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65f6d49b67-mdf7v"]
Sep 08 15:25:38.436720 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.436694    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-65f6d49b67-mdf7v"]
Sep 08 15:25:38.999873 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:38.999833    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:39.091060 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:39.091022    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:40.087275 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:40.087238    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:40.192661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:40.192625    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:40.432067 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:40.432033    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f16c0c75-f55b-4270-8835-fb71c63d5ad0" path="/var/lib/kubelet/pods/f16c0c75-f55b-4270-8835-fb71c63d5ad0/volumes"
Sep 08 15:25:49.171652 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:49.171618    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:55.429522 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:55.429467    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:55.546172 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:55.546142    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:55.828198 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:55.828106    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:56.242913 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:56.242871    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:57.178647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.178615    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:57.271824 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.271775    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:57.392513 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.392466    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:57.431467 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.431391    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7861-140416314687280"]
Sep 08 15:25:57.431834 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.431820    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16c0c75-f55b-4270-8835-fb71c63d5ad0" containerName="authorino"
Sep 08 15:25:57.431877 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.431836    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16c0c75-f55b-4270-8835-fb71c63d5ad0" containerName="authorino"
Sep 08 15:25:57.431913 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.431892    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16c0c75-f55b-4270-8835-fb71c63d5ad0" containerName="authorino"
Sep 08 15:25:57.435001 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.434986    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:25:57.437701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.437683    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kvpjx\""
Sep 08 15:25:57.444899 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.444880    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7861-140416314687280"]
Sep 08 15:25:57.597371 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.597333    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4fsw\" (UniqueName: \"kubernetes.io/projected/07966456-ffde-48f1-adb9-01fdd5748667-kube-api-access-b4fsw\") pod \"test-curl-7861-140416314687280\" (UID: \"07966456-ffde-48f1-adb9-01fdd5748667\") " pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:25:57.698483 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.698401    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4fsw\" (UniqueName: \"kubernetes.io/projected/07966456-ffde-48f1-adb9-01fdd5748667-kube-api-access-b4fsw\") pod \"test-curl-7861-140416314687280\" (UID: \"07966456-ffde-48f1-adb9-01fdd5748667\") " pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:25:57.716394 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.716364    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4fsw\" (UniqueName: \"kubernetes.io/projected/07966456-ffde-48f1-adb9-01fdd5748667-kube-api-access-b4fsw\") pod \"test-curl-7861-140416314687280\" (UID: \"07966456-ffde-48f1-adb9-01fdd5748667\") " pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:25:57.745420 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:57.745393    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:25:58.014388 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:58.014363    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7861-140416314687280"]
Sep 08 15:25:58.016827 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:25:58.016784    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07966456_ffde_48f1_adb9_01fdd5748667.slice/crio-6d62c65015adb0c17e2e82d8b009a967e29aa29561a06c75ffa39f3ba936c955 WatchSource:0}: Error finding container 6d62c65015adb0c17e2e82d8b009a967e29aa29561a06c75ffa39f3ba936c955: Status 404 returned error can't find the container with id 6d62c65015adb0c17e2e82d8b009a967e29aa29561a06c75ffa39f3ba936c955
Sep 08 15:25:58.497403 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:58.497372    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7861-140416314687280" event={"ID":"07966456-ffde-48f1-adb9-01fdd5748667","Type":"ContainerStarted","Data":"6d62c65015adb0c17e2e82d8b009a967e29aa29561a06c75ffa39f3ba936c955"}
Sep 08 15:25:58.927739 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:58.927707    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"]
Sep 08 15:25:58.931375 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:58.931354    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:58.933913 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:58.933889    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-af09fb66-kserve-self-signed-certs\""
Sep 08 15:25:58.944776 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:58.944753    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"]
Sep 08 15:25:58.962428 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:58.962399    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:59.111098 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.111064    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-dshm\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.111274 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.111121    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7bhh\" (UniqueName: \"kubernetes.io/projected/a1db9b37-f55a-4b92-848c-9427bfa957f6-kube-api-access-f7bhh\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.111274 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.111256    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1db9b37-f55a-4b92-848c-9427bfa957f6-tls-certs\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.111386 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.111322    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-model-cache\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.111436 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.111395    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-home\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.111436 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.111432    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-tmp-dir\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.137145 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.137113    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:25:59.193371 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.193295    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:25:59.212089 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.212060    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-dshm\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.212247 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.212102    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7bhh\" (UniqueName: \"kubernetes.io/projected/a1db9b37-f55a-4b92-848c-9427bfa957f6-kube-api-access-f7bhh\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.212247 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.212133    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1db9b37-f55a-4b92-848c-9427bfa957f6-tls-certs\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.212247 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.212177    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-model-cache\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.212247 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.212216    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-home\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.212476 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.212263    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-tmp-dir\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.212707 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.212658    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-tmp-dir\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.212835 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.212705    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-model-cache\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.212835 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.212730    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-home\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.214464 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.214442    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-dshm\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.214920 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.214899    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1db9b37-f55a-4b92-848c-9427bfa957f6-tls-certs\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.235349 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.235327    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7bhh\" (UniqueName: \"kubernetes.io/projected/a1db9b37-f55a-4b92-848c-9427bfa957f6-kube-api-access-f7bhh\") pod \"e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") " pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.245214 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.245194    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:25:59.296248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:25:59.296211    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:00.065778 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:00.065681    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"]
Sep 08 15:26:00.068459 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:26:00.068429    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1db9b37_f55a_4b92_848c_9427bfa957f6.slice/crio-ef1cda87f8714d788afdc6fab4270bb69a35c74ffac7827f280548184643b0f8 WatchSource:0}: Error finding container ef1cda87f8714d788afdc6fab4270bb69a35c74ffac7827f280548184643b0f8: Status 404 returned error can't find the container with id ef1cda87f8714d788afdc6fab4270bb69a35c74ffac7827f280548184643b0f8
Sep 08 15:26:00.479957 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:00.479926    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:00.515412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:00.515379    2575 generic.go:356] "Generic (PLEG): container finished" podID="07966456-ffde-48f1-adb9-01fdd5748667" containerID="6e4a4cb12a6481162feec4da4fa7d99e239f3a77335c0c4dc3694623a69dd617" exitCode=0
Sep 08 15:26:00.515595 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:00.515474    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7861-140416314687280" event={"ID":"07966456-ffde-48f1-adb9-01fdd5748667","Type":"ContainerDied","Data":"6e4a4cb12a6481162feec4da4fa7d99e239f3a77335c0c4dc3694623a69dd617"}
Sep 08 15:26:00.517737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:00.517707    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q" event={"ID":"a1db9b37-f55a-4b92-848c-9427bfa957f6","Type":"ContainerStarted","Data":"181a2a36530c1627c19131f78ac5fe46c3ced001864cba32b6ce55c8bc3c8f9e"}
Sep 08 15:26:00.517852 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:00.517746    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q" event={"ID":"a1db9b37-f55a-4b92-848c-9427bfa957f6","Type":"ContainerStarted","Data":"ef1cda87f8714d788afdc6fab4270bb69a35c74ffac7827f280548184643b0f8"}
Sep 08 15:26:00.551971 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:00.551910    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q" podStartSLOduration=2.551890305 podStartE2EDuration="2.551890305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:25:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:26:00.549884264 +0000 UTC m=+1008.635834306" watchObservedRunningTime="2026-09-08 15:26:00.551890305 +0000 UTC m=+1008.637840348"
Sep 08 15:26:00.551971 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:00.551959    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:00.955097 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:00.955064    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:01.055923 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.055883    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:01.167819 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.167768    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:01.649555 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.649526    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:26:01.672023 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.671999    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7861-140416314687280_07966456-ffde-48f1-adb9-01fdd5748667/test-curl-7861-140416314687280/0.log"
Sep 08 15:26:01.698919 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.698887    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7861-140416314687280"]
Sep 08 15:26:01.706537 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.706513    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7861-140416314687280"]
Sep 08 15:26:01.833377 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.833302    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7861-140416314687280"]
Sep 08 15:26:01.833760 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.833740    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07966456-ffde-48f1-adb9-01fdd5748667" containerName="test-curl-7861-140416314687280"
Sep 08 15:26:01.833852 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.833762    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="07966456-ffde-48f1-adb9-01fdd5748667" containerName="test-curl-7861-140416314687280"
Sep 08 15:26:01.833910 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.833893    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="07966456-ffde-48f1-adb9-01fdd5748667" containerName="test-curl-7861-140416314687280"
Sep 08 15:26:01.836860 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.836836    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4fsw\" (UniqueName: \"kubernetes.io/projected/07966456-ffde-48f1-adb9-01fdd5748667-kube-api-access-b4fsw\") pod \"07966456-ffde-48f1-adb9-01fdd5748667\" (UID: \"07966456-ffde-48f1-adb9-01fdd5748667\") "
Sep 08 15:26:01.836860 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.836851    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:26:01.839149 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.839123    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07966456-ffde-48f1-adb9-01fdd5748667-kube-api-access-b4fsw" (OuterVolumeSpecName: "kube-api-access-b4fsw") pod "07966456-ffde-48f1-adb9-01fdd5748667" (UID: "07966456-ffde-48f1-adb9-01fdd5748667"). InnerVolumeSpecName "kube-api-access-b4fsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:26:01.842232 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.842208    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7861-140416314687280"]
Sep 08 15:26:01.937676 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.937645    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g79p\" (UniqueName: \"kubernetes.io/projected/cb308123-49b5-4982-8fd8-251396711614-kube-api-access-7g79p\") pod \"test-curl-7861-140416314687280\" (UID: \"cb308123-49b5-4982-8fd8-251396711614\") " pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:26:01.937827 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:01.937751    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4fsw\" (UniqueName: \"kubernetes.io/projected/07966456-ffde-48f1-adb9-01fdd5748667-kube-api-access-b4fsw\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:26:02.039138 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:02.039105    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g79p\" (UniqueName: \"kubernetes.io/projected/cb308123-49b5-4982-8fd8-251396711614-kube-api-access-7g79p\") pod \"test-curl-7861-140416314687280\" (UID: \"cb308123-49b5-4982-8fd8-251396711614\") " pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:26:02.047361 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:02.047330    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g79p\" (UniqueName: \"kubernetes.io/projected/cb308123-49b5-4982-8fd8-251396711614-kube-api-access-7g79p\") pod \"test-curl-7861-140416314687280\" (UID: \"cb308123-49b5-4982-8fd8-251396711614\") " pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:26:02.155382 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:02.155355    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:26:02.429198 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:26:02.429169    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb308123_49b5_4982_8fd8_251396711614.slice/crio-22d881d0cf72340d386e83269c3025ca0056769dadb65ae8190be6fa62bef928 WatchSource:0}: Error finding container 22d881d0cf72340d386e83269c3025ca0056769dadb65ae8190be6fa62bef928: Status 404 returned error can't find the container with id 22d881d0cf72340d386e83269c3025ca0056769dadb65ae8190be6fa62bef928
Sep 08 15:26:02.432114 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:02.432089    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07966456-ffde-48f1-adb9-01fdd5748667" path="/var/lib/kubelet/pods/07966456-ffde-48f1-adb9-01fdd5748667/volumes"
Sep 08 15:26:02.432513 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:02.432472    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7861-140416314687280"]
Sep 08 15:26:02.528675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:02.528634    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7861-140416314687280" event={"ID":"cb308123-49b5-4982-8fd8-251396711614","Type":"ContainerStarted","Data":"22d881d0cf72340d386e83269c3025ca0056769dadb65ae8190be6fa62bef928"}
Sep 08 15:26:02.529875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:02.529855    2575 scope.go:117] "RemoveContainer" containerID="6e4a4cb12a6481162feec4da4fa7d99e239f3a77335c0c4dc3694623a69dd617"
Sep 08 15:26:02.529875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:02.529867    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:26:03.537371 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:03.537278    2575 generic.go:356] "Generic (PLEG): container finished" podID="cb308123-49b5-4982-8fd8-251396711614" containerID="60b932e9c6c9fd066bb75547077be955d7a751003d705f12425e2760077d1556" exitCode=0
Sep 08 15:26:03.537371 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:03.537336    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7861-140416314687280" event={"ID":"cb308123-49b5-4982-8fd8-251396711614","Type":"ContainerDied","Data":"60b932e9c6c9fd066bb75547077be955d7a751003d705f12425e2760077d1556"}
Sep 08 15:26:04.273622 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.273587    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:04.376642 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.376605    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:04.526869 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.526782    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:04.676850 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.676826    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:26:04.701150 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.701101    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7861-140416314687280_cb308123-49b5-4982-8fd8-251396711614/test-curl-7861-140416314687280/0.log"
Sep 08 15:26:04.736048 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.736011    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7861-140416314687280"]
Sep 08 15:26:04.744209 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.744180    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7861-140416314687280"]
Sep 08 15:26:04.761006 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.760982    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7g79p\" (UniqueName: \"kubernetes.io/projected/cb308123-49b5-4982-8fd8-251396711614-kube-api-access-7g79p\") pod \"cb308123-49b5-4982-8fd8-251396711614\" (UID: \"cb308123-49b5-4982-8fd8-251396711614\") "
Sep 08 15:26:04.763050 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.763026    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb308123-49b5-4982-8fd8-251396711614-kube-api-access-7g79p" (OuterVolumeSpecName: "kube-api-access-7g79p") pod "cb308123-49b5-4982-8fd8-251396711614" (UID: "cb308123-49b5-4982-8fd8-251396711614"). InnerVolumeSpecName "kube-api-access-7g79p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:26:04.800057 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.799988    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"]
Sep 08 15:26:04.800611 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.800593    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb308123-49b5-4982-8fd8-251396711614" containerName="test-curl-7861-140416314687280"
Sep 08 15:26:04.800690 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.800614    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb308123-49b5-4982-8fd8-251396711614" containerName="test-curl-7861-140416314687280"
Sep 08 15:26:04.800746 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.800726    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb308123-49b5-4982-8fd8-251396711614" containerName="test-curl-7861-140416314687280"
Sep 08 15:26:04.804084 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.804065    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.806747 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.806725    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-062f32f0-openshift-default-dockercfg-57prx\""
Sep 08 15:26:04.817826 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.817803    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"]
Sep 08 15:26:04.828341 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.828313    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:04.862680 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.862654    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-workload-certs\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.862795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.862690    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-envoy\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.862795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.862710    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-podinfo\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.862795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.862728    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-token\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.862795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.862745    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj9hd\" (UniqueName: \"kubernetes.io/projected/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-kube-api-access-qj9hd\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.862978 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.862815    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-credential-socket\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.862978 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.862859    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istiod-ca-cert\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.862978 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.862879    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-data\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.863127 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.862986    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-workload-socket\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.863127 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.863091    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g79p\" (UniqueName: \"kubernetes.io/projected/cb308123-49b5-4982-8fd8-251396711614-kube-api-access-7g79p\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:26:04.964157 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.964119    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-workload-certs\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.964352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.964168    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-envoy\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.964352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.964187    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-podinfo\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.964352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.964213    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-token\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.964352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.964236    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj9hd\" (UniqueName: \"kubernetes.io/projected/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-kube-api-access-qj9hd\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.964352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.964275    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-credential-socket\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.964352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.964318    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istiod-ca-cert\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.964352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.964344    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-data\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.964756 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.964417    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-workload-socket\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.964756 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.964632    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-workload-certs\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.964756 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.964745    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-credential-socket\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.964903 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.964762    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-data\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.964903 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.964831    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-workload-socket\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.965123 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.965104    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istiod-ca-cert\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.966584 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.966566    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-envoy\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.966778 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.966760    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-podinfo\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.972983 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.972958    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj9hd\" (UniqueName: \"kubernetes.io/projected/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-kube-api-access-qj9hd\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:04.973089 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:04.973040    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-token\") pod \"e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") " pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:05.115509 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.115462    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:05.281858 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.281826    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:05.395069 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.394964    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"]
Sep 08 15:26:05.396910 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.396880    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:05.398165 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:26:05.398140    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d26ee4_69e2_4095_a20e_3ffef2fa12c2.slice/crio-d94281ba22b7f1d60305cf61756bd647221adb6d7a4e21e06f25ab9ff3a33076 WatchSource:0}: Error finding container d94281ba22b7f1d60305cf61756bd647221adb6d7a4e21e06f25ab9ff3a33076: Status 404 returned error can't find the container with id d94281ba22b7f1d60305cf61756bd647221adb6d7a4e21e06f25ab9ff3a33076
Sep 08 15:26:05.400326 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.400302    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:05.400376 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.400362    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:05.400415 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.400391    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:05.550705 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.550661    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" event={"ID":"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2","Type":"ContainerStarted","Data":"b8193383d606c1173d0756d96a4588400641a7219d9c63034751e5e431337f60"}
Sep 08 15:26:05.551014 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.550737    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" event={"ID":"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2","Type":"ContainerStarted","Data":"d94281ba22b7f1d60305cf61756bd647221adb6d7a4e21e06f25ab9ff3a33076"}
Sep 08 15:26:05.552331 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.552310    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:26:05.552331 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.552310    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22d881d0cf72340d386e83269c3025ca0056769dadb65ae8190be6fa62bef928"
Sep 08 15:26:05.579065 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:26:05.579022    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7861-140416314687280\" is forbidden: User \"system:node:ip-10-0-132-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-76.ec2.internal' and this object" podUID="cb308123-49b5-4982-8fd8-251396711614" pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:26:05.586266 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.586216    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" podStartSLOduration=1.586194935 podStartE2EDuration="1.586194935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:26:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:26:05.577019053 +0000 UTC m=+1013.662969097" watchObservedRunningTime="2026-09-08 15:26:05.586194935 +0000 UTC m=+1013.672144979"
Sep 08 15:26:05.588414 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.588392    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"]
Sep 08 15:26:05.592540 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.592524    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.595940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.595894    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-3bc9dc-openshift-default-dockercfg-dh5zb\""
Sep 08 15:26:05.596047 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:26:05.595969    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7861-140416314687280\" is forbidden: User \"system:node:ip-10-0-132-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-76.ec2.internal' and this object" podUID="cb308123-49b5-4982-8fd8-251396711614" pod="opendatahub/test-curl-7861-140416314687280"
Sep 08 15:26:05.604302 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.604279    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"]
Sep 08 15:26:05.621963 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.621913    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:05.671096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.671006    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-workload-certs\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.671096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.671040    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wzrj\" (UniqueName: \"kubernetes.io/projected/c2f28e4b-890a-43c7-a1f2-152ec40553b2-kube-api-access-8wzrj\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.671325 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.671121    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-token\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.671325 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.671198    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-workload-socket\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.671325 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.671226    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-podinfo\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.671325 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.671277    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-credential-socket\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.671325 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.671315    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istiod-ca-cert\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.671631 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.671351    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-envoy\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.671631 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.671376    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-data\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.772162 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.772130    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-workload-socket\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.772661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.772171    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-podinfo\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.772661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.772215    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-credential-socket\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.772661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.772249    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istiod-ca-cert\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.772661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.772278    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-envoy\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.772661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.772306    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-data\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.772661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.772368    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-workload-certs\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.772661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.772397    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wzrj\" (UniqueName: \"kubernetes.io/projected/c2f28e4b-890a-43c7-a1f2-152ec40553b2-kube-api-access-8wzrj\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.772661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.772450    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-token\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.772661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.772572    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-workload-socket\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.772661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.772625    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-credential-socket\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.773335 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.772819    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-data\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.773335 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.772970    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istiod-ca-cert\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.773335 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.773080    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-workload-certs\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.775904 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.775879    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-envoy\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.776040 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.776015    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-podinfo\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.785738 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.785710    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wzrj\" (UniqueName: \"kubernetes.io/projected/c2f28e4b-890a-43c7-a1f2-152ec40553b2-kube-api-access-8wzrj\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.788137 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.788108    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-token\") pod \"e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") " pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:05.906059 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:05.906026    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:06.064720 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.064680    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:06.116396 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.116367    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:06.118222 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.118194    2575 patch_prober.go:28] interesting pod/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 08 15:26:06.118377 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.118251    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 08 15:26:06.199948 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.199921    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"]
Sep 08 15:26:06.202253 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:26:06.202221    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2f28e4b_890a_43c7_a1f2_152ec40553b2.slice/crio-e93ef088b62b61edcf177ac5909788ce212b8ec0c83769045b80c6a61b3916aa WatchSource:0}: Error finding container e93ef088b62b61edcf177ac5909788ce212b8ec0c83769045b80c6a61b3916aa: Status 404 returned error can't find the container with id e93ef088b62b61edcf177ac5909788ce212b8ec0c83769045b80c6a61b3916aa
Sep 08 15:26:06.204871 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.204840    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:06.204973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.204915    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:06.205035 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.204985    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:06.434622 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.434587    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb308123-49b5-4982-8fd8-251396711614" path="/var/lib/kubelet/pods/cb308123-49b5-4982-8fd8-251396711614/volumes"
Sep 08 15:26:06.558588 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.558504    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" event={"ID":"c2f28e4b-890a-43c7-a1f2-152ec40553b2","Type":"ContainerStarted","Data":"8696a126435a7925a47c3201dbc9e76036a771e740389a9720f1c811d4c1956e"}
Sep 08 15:26:06.558588 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.558557    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" event={"ID":"c2f28e4b-890a-43c7-a1f2-152ec40553b2","Type":"ContainerStarted","Data":"e93ef088b62b61edcf177ac5909788ce212b8ec0c83769045b80c6a61b3916aa"}
Sep 08 15:26:06.582726 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.582682    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" podStartSLOduration=1.58266513 podStartE2EDuration="1.58266513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:26:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:26:06.580786277 +0000 UTC m=+1014.666736321" watchObservedRunningTime="2026-09-08 15:26:06.58266513 +0000 UTC m=+1014.668615388"
Sep 08 15:26:06.906737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.906704    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:06.958772 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:06.958739    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:07.064714 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:07.064682    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:07.120708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:07.120679    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:07.563820 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:07.563792    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:07.564799 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:07.564774    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:07.912175 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:07.912148    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:08.185234 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:08.185158    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:08.282351 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:08.282316    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:08.382386 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:08.382349    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:08.550300 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:08.550213    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:08.568538 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:08.568481    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:08.569378 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:08.569358    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:08.688511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:08.688464    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:09.246099 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.246063    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:26:09.246615 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.246116    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:26:09.258959 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.258933    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:26:09.585432 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.585351    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:26:09.760521 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.760471    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:09.829330 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.829291    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:09.935998 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.935962    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"]
Sep 08 15:26:09.939463 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.939440    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:09.942321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.942301    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:26:09.942455 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.942367    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 08 15:26:09.942455 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.942386    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-3bc9dc-serving-cert\""
Sep 08 15:26:09.942455 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.942428    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 08 15:26:09.943319 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.943294    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hrbm8\""
Sep 08 15:26:09.952435 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.952418    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"]
Sep 08 15:26:09.977991 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:09.977965    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:10.018231 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.018198    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-maas-api-tls\") pod \"maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:10.018379 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.018240    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzf4p\" (UniqueName: \"kubernetes.io/projected/c7999d7e-8921-4542-ad1b-142cb114dc49-kube-api-access-xzf4p\") pod \"maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:10.018379 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.018351    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-metrics-certs\") pod \"maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:10.018480 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.018415    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7999d7e-8921-4542-ad1b-142cb114dc49-openshift-service-ca\") pod \"maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:10.078148 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.078112    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:10.119954 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.119922    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-metrics-certs\") pod \"maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:10.120103 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.119980    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7999d7e-8921-4542-ad1b-142cb114dc49-openshift-service-ca\") pod \"maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:10.120103 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.120053    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-maas-api-tls\") pod \"maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:10.120103 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.120086    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzf4p\" (UniqueName: \"kubernetes.io/projected/c7999d7e-8921-4542-ad1b-142cb114dc49-kube-api-access-xzf4p\") pod \"maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:10.120225 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:26:10.120192    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-serving-cert: secret "maas-api-e2e-autores-3bc9dc-serving-cert" not found
Sep 08 15:26:10.120274 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:26:10.120255    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-maas-api-tls podName:c7999d7e-8921-4542-ad1b-142cb114dc49 nodeName:}" failed. No retries permitted until 2026-09-08 15:26:10.620234459 +0000 UTC m=+1018.706184478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-maas-api-tls") pod "maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l" (UID: "c7999d7e-8921-4542-ad1b-142cb114dc49") : secret "maas-api-e2e-autores-3bc9dc-serving-cert" not found
Sep 08 15:26:10.120615 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.120598    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7999d7e-8921-4542-ad1b-142cb114dc49-openshift-service-ca\") pod \"maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:10.122418 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.122402    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-metrics-certs\") pod \"maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:10.128390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.128369    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzf4p\" (UniqueName: \"kubernetes.io/projected/c7999d7e-8921-4542-ad1b-142cb114dc49-kube-api-access-xzf4p\") pod \"maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:10.388676 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.388638    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:10.487455 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.487423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:10.574169 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.574137    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:10.625976 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.625936    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-maas-api-tls\") pod \"maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:10.628283 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.628250    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-maas-api-tls\") pod \"maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:10.851325 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:10.851239    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:11.123031 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:11.123011    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"]
Sep 08 15:26:11.125368 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:26:11.125343    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7999d7e_8921_4542_ad1b_142cb114dc49.slice/crio-7607de308bf87e6acff20f43dbf56788973bf79e89a313266e0f2e2745b944dc WatchSource:0}: Error finding container 7607de308bf87e6acff20f43dbf56788973bf79e89a313266e0f2e2745b944dc: Status 404 returned error can't find the container with id 7607de308bf87e6acff20f43dbf56788973bf79e89a313266e0f2e2745b944dc
Sep 08 15:26:11.548123 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:11.548044    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:26:11.584363 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:11.584328    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l" event={"ID":"c7999d7e-8921-4542-ad1b-142cb114dc49","Type":"ContainerStarted","Data":"7607de308bf87e6acff20f43dbf56788973bf79e89a313266e0f2e2745b944dc"}
Sep 08 15:26:13.597278 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:13.597235    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l" event={"ID":"c7999d7e-8921-4542-ad1b-142cb114dc49","Type":"ContainerStarted","Data":"b1e47a9d4f471d15cad0caf68ce8dfc6d6d8fec8898848622dea82a3f7e26b9e"}
Sep 08 15:26:13.597718 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:13.597307    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:13.614819 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:13.614766    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l" podStartSLOduration=2.97824553 podStartE2EDuration="4.614750712s" totalImagesPullingTime="1.636505182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:26:13.61398959 +0000 UTC m=+1021.699939633" watchObservedRunningTime="2026-09-08 15:26:13.614750712 +0000 UTC m=+1021.700700753"
Sep 08 15:26:14.299318 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:14.299284    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:14.478482 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:14.478442    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:14.608896 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:14.608858    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:14.694135 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:14.694099    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:16.248818 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:16.248757    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:16.341601 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:16.341565    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:16.475274 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:16.475232    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:17.360236 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:17.360204    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:18.980685 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:18.980649    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:19.614150 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:19.614119    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:26:21.665671 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:21.665637    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:22.558453 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:22.558378    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:22.565424 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:22.565393    2575 patch_prober.go:28] interesting pod/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 08 15:26:22.565594 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:22.565455    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 08 15:26:22.654722 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:22.654638    2575 generic.go:356] "Generic (PLEG): container finished" podID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerID="b8193383d606c1173d0756d96a4588400641a7219d9c63034751e5e431337f60" exitCode=0
Sep 08 15:26:22.654722 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:22.654703    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" event={"ID":"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2","Type":"ContainerDied","Data":"b8193383d606c1173d0756d96a4588400641a7219d9c63034751e5e431337f60"}
Sep 08 15:26:22.655270 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:22.655248    2575 scope.go:117] "RemoveContainer" containerID="b8193383d606c1173d0756d96a4588400641a7219d9c63034751e5e431337f60"
Sep 08 15:26:22.656630 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:22.656597    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:22.656748 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:22.656681    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:22.656748 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:22.656733    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:23.421010 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:23.420974    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:23.666182 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:23.666145    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" event={"ID":"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2","Type":"ContainerStarted","Data":"8eec5e6efe62e0b548968d08ef56d10c8ffdb5fb0e70b41edf08de5ba5c59385"}
Sep 08 15:26:23.741736 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:23.741635    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:23.882087 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:23.882052    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:23.999473 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:23.999372    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:24.107431 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:24.107401    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:24.116110 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:24.116074    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:24.116682 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:24.116650    2575 patch_prober.go:28] interesting pod/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 08 15:26:24.116896 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:24.116873    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 08 15:26:24.211453 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:24.211410    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:24.447219 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:24.447180    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:24.957263 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:24.957232    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:25.042154 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:25.042119    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:25.116226 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:25.116189    2575 patch_prober.go:28] interesting pod/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 08 15:26:25.116549 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:25.116481    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 08 15:26:26.116993 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:26.116951    2575 patch_prober.go:28] interesting pod/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 08 15:26:26.117652 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:26.117620    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 08 15:26:27.116378 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:27.116337    2575 patch_prober.go:28] interesting pod/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 08 15:26:27.116692 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:27.116660    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 08 15:26:28.116996 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:28.116947    2575 patch_prober.go:28] interesting pod/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 08 15:26:28.117481 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:28.117029    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 08 15:26:28.389198 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:28.389106    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:28.487175 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:28.487141    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:29.116320 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:29.116272    2575 patch_prober.go:28] interesting pod/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 08 15:26:29.116582 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:29.116345    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 08 15:26:29.379931 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:29.379827    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:30.120002 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:30.119973    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:30.120391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:30.120368    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:30.120943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:30.120925    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:30.123861 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:30.123833    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:30.228277 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:30.228134    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:30.406244 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:30.406155    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:30.434061 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:30.434033    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:26:30.676597 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:30.676482    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:30.762855 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:30.762813    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:31.162072 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:31.162035    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:31.663102 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:31.663022    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:31.709713 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:31.709689    2575 generic.go:356] "Generic (PLEG): container finished" podID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerID="8696a126435a7925a47c3201dbc9e76036a771e740389a9720f1c811d4c1956e" exitCode=0
Sep 08 15:26:31.709846 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:31.709773    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" event={"ID":"c2f28e4b-890a-43c7-a1f2-152ec40553b2","Type":"ContainerDied","Data":"8696a126435a7925a47c3201dbc9e76036a771e740389a9720f1c811d4c1956e"}
Sep 08 15:26:31.710246 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:31.710227    2575 scope.go:117] "RemoveContainer" containerID="8696a126435a7925a47c3201dbc9e76036a771e740389a9720f1c811d4c1956e"
Sep 08 15:26:31.711412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:31.711376    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:31.711512 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:31.711466    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:31.711562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:31.711546    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:31.858881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:31.858850    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:31.959858 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:31.959818    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:32.693324 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:32.693263    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:32.716660 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:32.716635    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" event={"ID":"c2f28e4b-890a-43c7-a1f2-152ec40553b2","Type":"ContainerStarted","Data":"3d203e22c6e509db550ea9bb7ba5501e1bf340f818b45ffe10df22307789571a"}
Sep 08 15:26:32.737059 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:32.737024    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68"]
Sep 08 15:26:32.739372 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:32.739351    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68"
Sep 08 15:26:32.743157 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:32.743137    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8qqvg\""
Sep 08 15:26:32.753361 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:32.753335    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68"]
Sep 08 15:26:32.867451 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:32.867416    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4f56e7-d5c8-4879-8531-8430017ac533-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-062f32f0-wjv68\" (UID: \"3b4f56e7-d5c8-4879-8531-8430017ac533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68"
Sep 08 15:26:32.906568 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:32.906542    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:32.906748 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:32.906730    2575 patch_prober.go:28] interesting pod/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 08 15:26:32.906799 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:32.906782    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 08 15:26:32.968436 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:32.968362    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4f56e7-d5c8-4879-8531-8430017ac533-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-062f32f0-wjv68\" (UID: \"3b4f56e7-d5c8-4879-8531-8430017ac533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68"
Sep 08 15:26:32.968952 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:32.968933    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4f56e7-d5c8-4879-8531-8430017ac533-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-062f32f0-wjv68\" (UID: \"3b4f56e7-d5c8-4879-8531-8430017ac533\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68"
Sep 08 15:26:33.054233 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:33.054190    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68"
Sep 08 15:26:33.358858 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:33.358828    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68"]
Sep 08 15:26:33.362621 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:26:33.362595    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b4f56e7_d5c8_4879_8531_8430017ac533.slice/crio-405932e56aa823d4a0dec126b0be79835c38098ce99a13e48306b559ce2fe605 WatchSource:0}: Error finding container 405932e56aa823d4a0dec126b0be79835c38098ce99a13e48306b559ce2fe605: Status 404 returned error can't find the container with id 405932e56aa823d4a0dec126b0be79835c38098ce99a13e48306b559ce2fe605
Sep 08 15:26:33.613353 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:33.613269    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:33.729878 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:33.729785    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68" event={"ID":"3b4f56e7-d5c8-4879-8531-8430017ac533","Type":"ContainerStarted","Data":"405932e56aa823d4a0dec126b0be79835c38098ce99a13e48306b559ce2fe605"}
Sep 08 15:26:33.907553 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:33.907434    2575 patch_prober.go:28] interesting pod/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 08 15:26:33.907553 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:33.907522    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 08 15:26:33.915138 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:33.915101    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:34.751742 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:34.751697    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:34.914941 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:34.914591    2575 patch_prober.go:28] interesting pod/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 08 15:26:34.914941 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:34.914643    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 08 15:26:35.914536 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:35.913102    2575 patch_prober.go:28] interesting pod/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 08 15:26:35.914536 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:35.913156    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 08 15:26:36.748667 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:36.748631    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68" event={"ID":"3b4f56e7-d5c8-4879-8531-8430017ac533","Type":"ContainerStarted","Data":"df5951839345fda087a2730496055e8cd569a4520a302b6fa95181ec3a48a5ea"}
Sep 08 15:26:36.764768 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:36.764719    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68" podStartSLOduration=2.255103532 podStartE2EDuration="4.764705128s" totalImagesPullingTime="2.509601596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:26:36.763907677 +0000 UTC m=+1044.849857720" watchObservedRunningTime="2026-09-08 15:26:36.764705128 +0000 UTC m=+1044.850655207"
Sep 08 15:26:36.906628 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:36.906597    2575 patch_prober.go:28] interesting pod/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 08 15:26:36.906860 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:36.906833    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 08 15:26:37.755671 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:37.755637    2575 generic.go:356] "Generic (PLEG): container finished" podID="3b4f56e7-d5c8-4879-8531-8430017ac533" containerID="df5951839345fda087a2730496055e8cd569a4520a302b6fa95181ec3a48a5ea" exitCode=0
Sep 08 15:26:37.756067 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:37.755714    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68" event={"ID":"3b4f56e7-d5c8-4879-8531-8430017ac533","Type":"ContainerDied","Data":"df5951839345fda087a2730496055e8cd569a4520a302b6fa95181ec3a48a5ea"}
Sep 08 15:26:37.907181 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:37.907143    2575 patch_prober.go:28] interesting pod/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 08 15:26:37.907469 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:37.907438    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 08 15:26:38.569224 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:38.569196    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:38.886387 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:38.886365    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68"
Sep 08 15:26:38.910321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:38.910297    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:38.911340 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:38.911318    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:26:38.948456 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:38.948430    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4f56e7-d5c8-4879-8531-8430017ac533-maas-api-service-ca\") pod \"3b4f56e7-d5c8-4879-8531-8430017ac533\" (UID: \"3b4f56e7-d5c8-4879-8531-8430017ac533\") "
Sep 08 15:26:38.948785 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:38.948761    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b4f56e7-d5c8-4879-8531-8430017ac533-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3b4f56e7-d5c8-4879-8531-8430017ac533" (UID: "3b4f56e7-d5c8-4879-8531-8430017ac533"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:26:38.948958 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:38.948940    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b4f56e7-d5c8-4879-8531-8430017ac533-maas-api-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:26:39.766811 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:39.766785    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68"
Sep 08 15:26:39.766811 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:39.766797    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68" event={"ID":"3b4f56e7-d5c8-4879-8531-8430017ac533","Type":"ContainerDied","Data":"405932e56aa823d4a0dec126b0be79835c38098ce99a13e48306b559ce2fe605"}
Sep 08 15:26:39.767060 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:39.766844    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="405932e56aa823d4a0dec126b0be79835c38098ce99a13e48306b559ce2fe605"
Sep 08 15:26:45.738221 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:45.738186    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:26:48.747192 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.747164    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"]
Sep 08 15:26:48.747667 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.747651    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b4f56e7-d5c8-4879-8531-8430017ac533" containerName="revoke-keys"
Sep 08 15:26:48.747714 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.747669    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b4f56e7-d5c8-4879-8531-8430017ac533" containerName="revoke-keys"
Sep 08 15:26:48.747753 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.747743    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b4f56e7-d5c8-4879-8531-8430017ac533" containerName="revoke-keys"
Sep 08 15:26:48.752521 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.752481    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.755443 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.755419    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-af09fb66-kserve-self-signed-certs\""
Sep 08 15:26:48.759845 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.759538    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"]
Sep 08 15:26:48.842671 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.842646    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6276r\" (UniqueName: \"kubernetes.io/projected/0aad21b6-b2cd-43fe-a741-226552691d82-kube-api-access-6276r\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.842804 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.842702    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-model-cache\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.842804 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.842751    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aad21b6-b2cd-43fe-a741-226552691d82-tls-certs\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.842804 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.842766    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-dshm\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.842909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.842836    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-home\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.842909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.842880    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-tmp-dir\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.943968 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.943936    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6276r\" (UniqueName: \"kubernetes.io/projected/0aad21b6-b2cd-43fe-a741-226552691d82-kube-api-access-6276r\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.944135 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.944011    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-model-cache\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.944135 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.944067    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aad21b6-b2cd-43fe-a741-226552691d82-tls-certs\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.944135 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.944092    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-dshm\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.944305 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.944136    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-home\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.944305 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.944178    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-tmp-dir\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.944480 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.944454    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-model-cache\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.944588 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.944547    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-tmp-dir\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.944851 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.944830    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-home\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.946709 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.946686    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-dshm\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.947285 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.947243    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aad21b6-b2cd-43fe-a741-226552691d82-tls-certs\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:48.952276 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.952187    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:48.957416 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:48.957387    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6276r\" (UniqueName: \"kubernetes.io/projected/0aad21b6-b2cd-43fe-a741-226552691d82-kube-api-access-6276r\") pod \"e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") " pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:49.038924 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.038828    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:49.062598 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.062570    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:49.121990 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.121925    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:49.202823 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.202792    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68"]
Sep 08 15:26:49.205011 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.204983    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:49.205556 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.205536    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-062f32f0-wjv68"]
Sep 08 15:26:49.332605 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.332515    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"]
Sep 08 15:26:49.332769 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.332732    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy" containerID="cri-o://8eec5e6efe62e0b548968d08ef56d10c8ffdb5fb0e70b41edf08de5ba5c59385" gracePeriod=30
Sep 08 15:26:49.337704 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.337430    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"]
Sep 08 15:26:49.340479 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:26:49.340451    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0aad21b6_b2cd_43fe_a741_226552691d82.slice/crio-f99ebfed92a323a3e9b338a870d0dda926f978092d1fb0b74e97eb3f7f6c26f3 WatchSource:0}: Error finding container f99ebfed92a323a3e9b338a870d0dda926f978092d1fb0b74e97eb3f7f6c26f3: Status 404 returned error can't find the container with id f99ebfed92a323a3e9b338a870d0dda926f978092d1fb0b74e97eb3f7f6c26f3
Sep 08 15:26:49.678426 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.678393    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:49.787867 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.787835    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:49.818675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.818653    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm" event={"ID":"0aad21b6-b2cd-43fe-a741-226552691d82","Type":"ContainerStarted","Data":"38a3f37b21f126481df8908d8bb2f7a6c71ef86adc34b80b9cfe8d8d55e21610"}
Sep 08 15:26:49.818798 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.818688    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm" event={"ID":"0aad21b6-b2cd-43fe-a741-226552691d82","Type":"ContainerStarted","Data":"f99ebfed92a323a3e9b338a870d0dda926f978092d1fb0b74e97eb3f7f6c26f3"}
Sep 08 15:26:49.846109 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.846057    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm" podStartSLOduration=1.8460373319999999 podStartE2EDuration="1.846037332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:26:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:26:49.843677213 +0000 UTC m=+1057.929627278" watchObservedRunningTime="2026-09-08 15:26:49.846037332 +0000 UTC m=+1057.931987374"
Sep 08 15:26:49.880905 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:49.880875    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:50.034222 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.034131    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:50.356796 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.356766    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"]
Sep 08 15:26:50.360343 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.360327    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.362724 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.362701    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-d1aa3ad8-gw-openshift-default-dockercfg-q57xn\""
Sep 08 15:26:50.373799 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.373776    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"]
Sep 08 15:26:50.390648 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.390626    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:50.434322 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.434295    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b4f56e7-d5c8-4879-8531-8430017ac533" path="/var/lib/kubelet/pods/3b4f56e7-d5c8-4879-8531-8430017ac533/volumes"
Sep 08 15:26:50.459555 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.459535    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-workload-certs\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.459641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.459562    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7gkl\" (UniqueName: \"kubernetes.io/projected/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-kube-api-access-n7gkl\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.459641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.459585    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-credential-socket\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.459641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.459603    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istiod-ca-cert\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.459742 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.459665    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-workload-socket\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.459742 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.459699    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-envoy\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.459742 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.459728    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-data\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.459836 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.459753    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-token\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.459886 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.459835    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-podinfo\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.540619 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.540595    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:50.560552 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.560528    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-workload-socket\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.560649 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.560573    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-envoy\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.560722 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.560701    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-data\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.560774 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.560760    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-token\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.561090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.560872    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-workload-socket\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.561090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.561087    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-data\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.561251 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.561128    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-podinfo\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.561251 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.561174    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-workload-certs\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.561251 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.561206    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7gkl\" (UniqueName: \"kubernetes.io/projected/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-kube-api-access-n7gkl\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.561251 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.561244    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-credential-socket\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.561449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.561274    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istiod-ca-cert\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.561528 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.561512    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-workload-certs\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.561633 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.561615    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-credential-socket\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.561972 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.561950    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istiod-ca-cert\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.562930 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.562915    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-envoy\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.563356 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.563341    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-podinfo\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.569460 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.569440    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-token\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.569988 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.569958    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7gkl\" (UniqueName: \"kubernetes.io/projected/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-kube-api-access-n7gkl\") pod \"e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") " pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.670309 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.670235    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:50.947748 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.947722    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"]
Sep 08 15:26:50.949720 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:26:50.949699    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd28511e_c0f8_4e14_9828_5cb5d8bdf537.slice/crio-06310fcf2c6da80aa183f2307397daf96e021055d7dd8101446c99bc9674157a WatchSource:0}: Error finding container 06310fcf2c6da80aa183f2307397daf96e021055d7dd8101446c99bc9674157a: Status 404 returned error can't find the container with id 06310fcf2c6da80aa183f2307397daf96e021055d7dd8101446c99bc9674157a
Sep 08 15:26:50.951785 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.951752    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:50.951884 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.951825    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:50.951884 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:50.951867    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:26:51.513118 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:51.513060    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:51.837296 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:51.837188    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg" event={"ID":"dd28511e-c0f8-4e14-9828-5cb5d8bdf537","Type":"ContainerStarted","Data":"8d279e8eb959692e1df5fc29766eac1b3f7c0c5bdfce513a7f6815e244a8a84e"}
Sep 08 15:26:51.837296 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:51.837253    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg" event={"ID":"dd28511e-c0f8-4e14-9828-5cb5d8bdf537","Type":"ContainerStarted","Data":"06310fcf2c6da80aa183f2307397daf96e021055d7dd8101446c99bc9674157a"}
Sep 08 15:26:51.858531 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:51.858468    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg" podStartSLOduration=1.858451773 podStartE2EDuration="1.858451773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:26:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:26:51.856435206 +0000 UTC m=+1059.942385250" watchObservedRunningTime="2026-09-08 15:26:51.858451773 +0000 UTC m=+1059.944401813"
Sep 08 15:26:52.671167 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:52.671138    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:52.675972 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:52.675948    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:52.841673 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:52.841644    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:52.842714 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:52.842687    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:26:53.229376 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.229341    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:53.330690 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.330662    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:53.437687 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.437653    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:53.438553 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.438529    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"]
Sep 08 15:26:53.442082 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.442062    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:53.444884 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.444864    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-d1aa3ad8-serving-cert\""
Sep 08 15:26:53.466616 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.466595    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"]
Sep 08 15:26:53.491463 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.491412    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-openshift-service-ca\") pod \"maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:53.491463 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.491441    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-maas-api-tls\") pod \"maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:53.491636 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.491476    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-metrics-certs\") pod \"maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:53.491636 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.491544    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsw7j\" (UniqueName: \"kubernetes.io/projected/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-kube-api-access-rsw7j\") pod \"maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:53.592615 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.592589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-openshift-service-ca\") pod \"maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:53.592772 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.592624    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-maas-api-tls\") pod \"maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:53.592829 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.592809    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-metrics-certs\") pod \"maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:53.592881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.592857    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsw7j\" (UniqueName: \"kubernetes.io/projected/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-kube-api-access-rsw7j\") pod \"maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:53.593404 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.593376    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-openshift-service-ca\") pod \"maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:53.593639 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.593602    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:53.595123 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.595101    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-maas-api-tls\") pod \"maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:53.595284 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.595264    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-metrics-certs\") pod \"maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:53.610572 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.610550    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsw7j\" (UniqueName: \"kubernetes.io/projected/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-kube-api-access-rsw7j\") pod \"maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:53.701616 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.701593    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:53.754196 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:53.754140    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:54.024681 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.024615    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"]
Sep 08 15:26:54.026894 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:26:54.026868    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb16d3a4c_5d48_479e_9cbe_53852ef8b35a.slice/crio-39977bac0669e496b8312dae36954deb9a43bd9a44094030119f23fdf67e5c52 WatchSource:0}: Error finding container 39977bac0669e496b8312dae36954deb9a43bd9a44094030119f23fdf67e5c52: Status 404 returned error can't find the container with id 39977bac0669e496b8312dae36954deb9a43bd9a44094030119f23fdf67e5c52
Sep 08 15:26:54.101194 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.101159    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:54.374128 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.374095    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:26:54.559954 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.559930    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:54.704385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.704360    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qj9hd\" (UniqueName: \"kubernetes.io/projected/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-kube-api-access-qj9hd\") pod \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") "
Sep 08 15:26:54.704719 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.704400    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-credential-socket\") pod \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") "
Sep 08 15:26:54.704719 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.704421    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istiod-ca-cert\") pod \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") "
Sep 08 15:26:54.704719 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.704479    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-workload-certs\") pod \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") "
Sep 08 15:26:54.704719 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.704515    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-token\") pod \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") "
Sep 08 15:26:54.704719 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.704550    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-workload-socket\") pod \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") "
Sep 08 15:26:54.704719 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.704581    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-data\") pod \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") "
Sep 08 15:26:54.704719 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.704635    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-envoy\") pod \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") "
Sep 08 15:26:54.704719 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.704672    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-podinfo\") pod \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\" (UID: \"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2\") "
Sep 08 15:26:54.704719 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.704705    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" (UID: "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:26:54.705220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.704872    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" (UID: "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:26:54.705220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.704860    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" (UID: "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:26:54.705220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.704931    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" (UID: "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:26:54.705220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.705018    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" (UID: "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:26:54.705220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.705088    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:26:54.705220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.705110    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:26:54.705220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.705125    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:26:54.705220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.705137    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:26:54.706748 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.706722    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" (UID: "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:26:54.706835 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.706760    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-kube-api-access-qj9hd" (OuterVolumeSpecName: "kube-api-access-qj9hd") pod "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" (UID: "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2"). InnerVolumeSpecName "kube-api-access-qj9hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:26:54.706835 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.706790    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" (UID: "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:26:54.706835 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.706823    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" (UID: "c4d26ee4-69e2-4095-a20e-3ffef2fa12c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:26:54.806149 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.806124    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:26:54.806149 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.806146    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:26:54.806311 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.806155    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj9hd\" (UniqueName: \"kubernetes.io/projected/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-kube-api-access-qj9hd\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:26:54.806311 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.806163    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:26:54.806311 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.806171    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:26:54.854187 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.854157    2575 generic.go:356] "Generic (PLEG): container finished" podID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerID="8eec5e6efe62e0b548968d08ef56d10c8ffdb5fb0e70b41edf08de5ba5c59385" exitCode=0
Sep 08 15:26:54.854314 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.854201    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"
Sep 08 15:26:54.854314 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.854229    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" event={"ID":"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2","Type":"ContainerDied","Data":"8eec5e6efe62e0b548968d08ef56d10c8ffdb5fb0e70b41edf08de5ba5c59385"}
Sep 08 15:26:54.854314 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.854280    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt" event={"ID":"c4d26ee4-69e2-4095-a20e-3ffef2fa12c2","Type":"ContainerDied","Data":"d94281ba22b7f1d60305cf61756bd647221adb6d7a4e21e06f25ab9ff3a33076"}
Sep 08 15:26:54.854314 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.854310    2575 scope.go:117] "RemoveContainer" containerID="8eec5e6efe62e0b548968d08ef56d10c8ffdb5fb0e70b41edf08de5ba5c59385"
Sep 08 15:26:54.856181 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.856105    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt" event={"ID":"b16d3a4c-5d48-479e-9cbe-53852ef8b35a","Type":"ContainerStarted","Data":"711462de2006f6f6376f1b18810f0f110ce5e16005f3ab6050be5fd6eb4c2413"}
Sep 08 15:26:54.856286 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.856181    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt" event={"ID":"b16d3a4c-5d48-479e-9cbe-53852ef8b35a","Type":"ContainerStarted","Data":"39977bac0669e496b8312dae36954deb9a43bd9a44094030119f23fdf67e5c52"}
Sep 08 15:26:54.856286 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.856231    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:26:54.866559 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.866536    2575 scope.go:117] "RemoveContainer" containerID="b8193383d606c1173d0756d96a4588400641a7219d9c63034751e5e431337f60"
Sep 08 15:26:54.875545 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.875528    2575 scope.go:117] "RemoveContainer" containerID="8eec5e6efe62e0b548968d08ef56d10c8ffdb5fb0e70b41edf08de5ba5c59385"
Sep 08 15:26:54.875807 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:26:54.875786    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eec5e6efe62e0b548968d08ef56d10c8ffdb5fb0e70b41edf08de5ba5c59385\": container with ID starting with 8eec5e6efe62e0b548968d08ef56d10c8ffdb5fb0e70b41edf08de5ba5c59385 not found: ID does not exist" containerID="8eec5e6efe62e0b548968d08ef56d10c8ffdb5fb0e70b41edf08de5ba5c59385"
Sep 08 15:26:54.875872 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.875822    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eec5e6efe62e0b548968d08ef56d10c8ffdb5fb0e70b41edf08de5ba5c59385"} err="failed to get container status \"8eec5e6efe62e0b548968d08ef56d10c8ffdb5fb0e70b41edf08de5ba5c59385\": rpc error: code = NotFound desc = could not find container \"8eec5e6efe62e0b548968d08ef56d10c8ffdb5fb0e70b41edf08de5ba5c59385\": container with ID starting with 8eec5e6efe62e0b548968d08ef56d10c8ffdb5fb0e70b41edf08de5ba5c59385 not found: ID does not exist"
Sep 08 15:26:54.875872 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.875838    2575 scope.go:117] "RemoveContainer" containerID="b8193383d606c1173d0756d96a4588400641a7219d9c63034751e5e431337f60"
Sep 08 15:26:54.876060 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:26:54.876041    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8193383d606c1173d0756d96a4588400641a7219d9c63034751e5e431337f60\": container with ID starting with b8193383d606c1173d0756d96a4588400641a7219d9c63034751e5e431337f60 not found: ID does not exist" containerID="b8193383d606c1173d0756d96a4588400641a7219d9c63034751e5e431337f60"
Sep 08 15:26:54.876099 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.876065    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8193383d606c1173d0756d96a4588400641a7219d9c63034751e5e431337f60"} err="failed to get container status \"b8193383d606c1173d0756d96a4588400641a7219d9c63034751e5e431337f60\": rpc error: code = NotFound desc = could not find container \"b8193383d606c1173d0756d96a4588400641a7219d9c63034751e5e431337f60\": container with ID starting with b8193383d606c1173d0756d96a4588400641a7219d9c63034751e5e431337f60 not found: ID does not exist"
Sep 08 15:26:54.881992 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.881954    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt" podStartSLOduration=1.154449872 podStartE2EDuration="1.881941781s" totalImagesPullingTime="727.491909ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:26:54.88080026 +0000 UTC m=+1062.966750299" watchObservedRunningTime="2026-09-08 15:26:54.881941781 +0000 UTC m=+1062.967891822"
Sep 08 15:26:54.899975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.899955    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"]
Sep 08 15:26:54.904617 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:54.904596    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-062f32f0-openshift-default-6df7678cf8-2x5jt"]
Sep 08 15:26:56.431432 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:56.431400    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" path="/var/lib/kubelet/pods/c4d26ee4-69e2-4095-a20e-3ffef2fa12c2/volumes"
Sep 08 15:26:59.063323 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:59.063292    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:59.063668 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:59.063333    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:59.075127 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:59.075103    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:26:59.892220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:26:59.892187    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:27:00.866736 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:00.866712    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:27:04.440595 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:04.440561    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:27:13.960167 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:13.960133    2575 generic.go:356] "Generic (PLEG): container finished" podID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerID="8d279e8eb959692e1df5fc29766eac1b3f7c0c5bdfce513a7f6815e244a8a84e" exitCode=0
Sep 08 15:27:13.960538 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:13.960230    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg" event={"ID":"dd28511e-c0f8-4e14-9828-5cb5d8bdf537","Type":"ContainerDied","Data":"8d279e8eb959692e1df5fc29766eac1b3f7c0c5bdfce513a7f6815e244a8a84e"}
Sep 08 15:27:13.960751 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:13.960736    2575 scope.go:117] "RemoveContainer" containerID="8d279e8eb959692e1df5fc29766eac1b3f7c0c5bdfce513a7f6815e244a8a84e"
Sep 08 15:27:13.961794 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:13.961768    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:27:13.961908 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:13.961846    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:27:13.961908 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:13.961896    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:27:14.966680 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:14.966647    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg" event={"ID":"dd28511e-c0f8-4e14-9828-5cb5d8bdf537","Type":"ContainerStarted","Data":"eeb22375c7fc021494ad7a3bf03cb9368c1c3d099ac1eb93caeff632ac52c47d"}
Sep 08 15:27:15.671260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:15.671224    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:27:15.671461 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:15.671437    2575 patch_prober.go:28] interesting pod/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 08 15:27:15.671571 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:15.671507    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg" podUID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 08 15:27:15.833845 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:15.833813    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:27:16.671276 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:16.671222    2575 patch_prober.go:28] interesting pod/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 08 15:27:16.671641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:16.671316    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg" podUID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 08 15:27:17.671558 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:17.671522    2575 patch_prober.go:28] interesting pod/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 08 15:27:17.671915 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:17.671577    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg" podUID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 08 15:27:18.671513 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.671466    2575 patch_prober.go:28] interesting pod/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 08 15:27:18.671684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.671542    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg" podUID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 08 15:27:18.832303 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.832259    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"]
Sep 08 15:27:18.832730 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.832712    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy"
Sep 08 15:27:18.832730 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.832729    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy"
Sep 08 15:27:18.832894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.832811    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy"
Sep 08 15:27:18.832894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.832822    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy"
Sep 08 15:27:18.834937 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.834919    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.839589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.839566    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-3bc9dc\"/\"kube-root-ca.crt\""
Sep 08 15:27:18.839589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.839579    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-3bc9dc\"/\"autores-model-3bc9dc-kserve-self-signed-certs\""
Sep 08 15:27:18.839890 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.839874    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-3bc9dc\"/\"default-dockercfg-89wrf\""
Sep 08 15:27:18.840068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.840054    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-3bc9dc\"/\"openshift-service-ca.crt\""
Sep 08 15:27:18.840157 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.840139    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-tmp-dir\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.840211 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.840195    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czwjw\" (UniqueName: \"kubernetes.io/projected/2546172d-8096-4569-a43f-cffe19560fa0-kube-api-access-czwjw\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.840263 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.840228    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.840263 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.840245    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-model-cache\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.840364 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.840284    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-home\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.840364 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.840304    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-dshm\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.846183 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.846160    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"]
Sep 08 15:27:18.876524 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.876479    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:18.941808 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.941728    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-tmp-dir\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.941808 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.941802    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czwjw\" (UniqueName: \"kubernetes.io/projected/2546172d-8096-4569-a43f-cffe19560fa0-kube-api-access-czwjw\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.941994 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.941852    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.941994 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.941878    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-model-cache\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.941994 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.941933    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-home\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.941994 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.941965    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-dshm\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.942185 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.942163    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-tmp-dir\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.942286 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.942263    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-model-cache\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.942323 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.942299    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-home\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.944037 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.944013    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-dshm\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.944218 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.944203    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:18.949849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:18.949828    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czwjw\" (UniqueName: \"kubernetes.io/projected/2546172d-8096-4569-a43f-cffe19560fa0-kube-api-access-czwjw\") pod \"autores-model-3bc9dc-kserve-5b7d748bc5-t4drf\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") " pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:19.146711 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:19.146674    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:19.442352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:19.442322    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"]
Sep 08 15:27:19.444034 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:27:19.444010    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2546172d_8096_4569_a43f_cffe19560fa0.slice/crio-a9a112c1d3eebca532bc0ee871c580ac4b412658fbd32dd1f329b0581f15aa77 WatchSource:0}: Error finding container a9a112c1d3eebca532bc0ee871c580ac4b412658fbd32dd1f329b0581f15aa77: Status 404 returned error can't find the container with id a9a112c1d3eebca532bc0ee871c580ac4b412658fbd32dd1f329b0581f15aa77
Sep 08 15:27:19.670746 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:19.670714    2575 patch_prober.go:28] interesting pod/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 08 15:27:19.670894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:19.670762    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg" podUID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 08 15:27:19.994887 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:19.994847    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf" event={"ID":"2546172d-8096-4569-a43f-cffe19560fa0","Type":"ContainerStarted","Data":"67e2314e4ede69ec183e7f022294101506b692e586f29937d0cbf3fa6df425fb"}
Sep 08 15:27:19.995277 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:19.994898    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf" event={"ID":"2546172d-8096-4569-a43f-cffe19560fa0","Type":"ContainerStarted","Data":"a9a112c1d3eebca532bc0ee871c580ac4b412658fbd32dd1f329b0581f15aa77"}
Sep 08 15:27:20.015023 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:20.014961    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf" podStartSLOduration=2.014939635 podStartE2EDuration="2.014939635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:27:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:27:20.01292389 +0000 UTC m=+1088.098873932" watchObservedRunningTime="2026-09-08 15:27:20.014939635 +0000 UTC m=+1088.100889678"
Sep 08 15:27:20.675224 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:20.675193    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:27:20.675550 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:20.675525    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:27:20.676279 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:20.676263    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:27:20.933650 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:20.933573    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"]
Sep 08 15:27:20.955898 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:20.955874    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:29.147135 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.147102    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:29.147631 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.147179    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:29.159367 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.159348    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:29.628553 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.628521    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"]
Sep 08 15:27:29.628972 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.628955    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy"
Sep 08 15:27:29.629057 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.628975    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4d26ee4-69e2-4095-a20e-3ffef2fa12c2" containerName="istio-proxy"
Sep 08 15:27:29.631453 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.631431    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.634178 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.634160    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d8e721\"/\"default-dockercfg-2n6z4\""
Sep 08 15:27:29.634386 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.634366    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d8e721\"/\"kube-root-ca.crt\""
Sep 08 15:27:29.634527 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.634505    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d8e721\"/\"openshift-service-ca.crt\""
Sep 08 15:27:29.635450 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.635432    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d8e721\"/\"test-model-d8e721-kserve-self-signed-certs\""
Sep 08 15:27:29.643283 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.643262    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"]
Sep 08 15:27:29.674310 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.674274    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:29.735025 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.735006    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-tls-certs\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.735141 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.735052    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-home\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.735141 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.735130    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-dshm\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.735265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.735159    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-tmp-dir\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.735265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.735223    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-model-cache\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.735265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.735259    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4724\" (UniqueName: \"kubernetes.io/projected/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-kube-api-access-h4724\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.836210 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.836184    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-tls-certs\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.836376 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.836228    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-home\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.836376 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.836286    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-dshm\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.836376 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.836313    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-tmp-dir\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.836376 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.836367    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-model-cache\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.836605 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.836403    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4724\" (UniqueName: \"kubernetes.io/projected/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-kube-api-access-h4724\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.836856 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.836829    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-home\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.836969 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.836857    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-model-cache\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.836969 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.836897    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-tmp-dir\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.838513 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.838477    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-dshm\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.838873 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.838856    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-tls-certs\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.845077 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.845054    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4724\" (UniqueName: \"kubernetes.io/projected/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-kube-api-access-h4724\") pod \"test-model-d8e721-kserve-776d95f4dc-lx4sd\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") " pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:29.941530 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:29.941460    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:30.069137 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:30.069107    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:30.211726 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:30.211634    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"]
Sep 08 15:27:30.215393 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:27:30.215364    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa7ea7b7_ff49_4ed4_a448_2f451dbab064.slice/crio-f855fa4c002239fce8a6ba1702c1cec70c071b56084ff08d50ee9aa4ea039f78 WatchSource:0}: Error finding container f855fa4c002239fce8a6ba1702c1cec70c071b56084ff08d50ee9aa4ea039f78: Status 404 returned error can't find the container with id f855fa4c002239fce8a6ba1702c1cec70c071b56084ff08d50ee9aa4ea039f78
Sep 08 15:27:31.064205 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:31.064171    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd" event={"ID":"aa7ea7b7-ff49-4ed4-a448-2f451dbab064","Type":"ContainerStarted","Data":"b0fb50168f7d309261963cf055119213c3af2a088d000e0ad1c8740eb5ac9580"}
Sep 08 15:27:31.064356 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:31.064215    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd" event={"ID":"aa7ea7b7-ff49-4ed4-a448-2f451dbab064","Type":"ContainerStarted","Data":"f855fa4c002239fce8a6ba1702c1cec70c071b56084ff08d50ee9aa4ea039f78"}
Sep 08 15:27:31.083861 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:31.083819    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd" podStartSLOduration=2.083805877 podStartE2EDuration="2.083805877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:27:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:27:31.081843066 +0000 UTC m=+1099.167793098" watchObservedRunningTime="2026-09-08 15:27:31.083805877 +0000 UTC m=+1099.169755918"
Sep 08 15:27:31.731834 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:31.731802    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"]
Sep 08 15:27:31.752581 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:31.752552    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:32.078083 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:32.077992    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:32.158377 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:32.158344    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-self-signed-certs: secret "autores-model-3bc9dc-kserve-self-signed-certs" not found
Sep 08 15:27:32.158556 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:32.158414    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs podName:2546172d-8096-4569-a43f-cffe19560fa0 nodeName:}" failed. No retries permitted until 2026-09-08 15:27:32.65839442 +0000 UTC m=+1100.744344453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs") pod "autores-model-3bc9dc-kserve-5b7d748bc5-t4drf" (UID: "2546172d-8096-4569-a43f-cffe19560fa0") : secret "autores-model-3bc9dc-kserve-self-signed-certs" not found
Sep 08 15:27:32.663578 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:32.663549    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-self-signed-certs: secret "autores-model-3bc9dc-kserve-self-signed-certs" not found
Sep 08 15:27:32.663749 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:32.663611    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs podName:2546172d-8096-4569-a43f-cffe19560fa0 nodeName:}" failed. No retries permitted until 2026-09-08 15:27:33.663595233 +0000 UTC m=+1101.749545256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs") pod "autores-model-3bc9dc-kserve-5b7d748bc5-t4drf" (UID: "2546172d-8096-4569-a43f-cffe19560fa0") : secret "autores-model-3bc9dc-kserve-self-signed-certs" not found
Sep 08 15:27:33.636143 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:33.636109    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"]
Sep 08 15:27:33.666513 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:33.666458    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:33.671760 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:33.671741    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-self-signed-certs: secret "autores-model-3bc9dc-kserve-self-signed-certs" not found
Sep 08 15:27:33.671858 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:33.671791    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs podName:2546172d-8096-4569-a43f-cffe19560fa0 nodeName:}" failed. No retries permitted until 2026-09-08 15:27:35.671779357 +0000 UTC m=+1103.757729376 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs") pod "autores-model-3bc9dc-kserve-5b7d748bc5-t4drf" (UID: "2546172d-8096-4569-a43f-cffe19560fa0") : secret "autores-model-3bc9dc-kserve-self-signed-certs" not found
Sep 08 15:27:34.557516 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:34.557465    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:34.677130 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:34.677099    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:27:35.316916 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.316886    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"]
Sep 08 15:27:35.317169 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.317135    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt" podUID="b16d3a4c-5d48-479e-9cbe-53852ef8b35a" containerName="maas-api" containerID="cri-o://711462de2006f6f6376f1b18810f0f110ce5e16005f3ab6050be5fd6eb4c2413" gracePeriod=30
Sep 08 15:27:35.353076 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.353049    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:35.536851 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.536805    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:35.574456 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.574410    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:27:35.624711 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.624685    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:35.691944 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.691916    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-maas-api-tls\") pod \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") "
Sep 08 15:27:35.692309 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.692032    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-metrics-certs\") pod \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") "
Sep 08 15:27:35.692309 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.692072    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsw7j\" (UniqueName: \"kubernetes.io/projected/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-kube-api-access-rsw7j\") pod \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") "
Sep 08 15:27:35.692309 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.692173    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-openshift-service-ca\") pod \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\" (UID: \"b16d3a4c-5d48-479e-9cbe-53852ef8b35a\") "
Sep 08 15:27:35.692596 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.692560    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b16d3a4c-5d48-479e-9cbe-53852ef8b35a" (UID: "b16d3a4c-5d48-479e-9cbe-53852ef8b35a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:27:35.692768 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:35.692609    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-self-signed-certs: secret "autores-model-3bc9dc-kserve-self-signed-certs" not found
Sep 08 15:27:35.692768 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:35.692711    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs podName:2546172d-8096-4569-a43f-cffe19560fa0 nodeName:}" failed. No retries permitted until 2026-09-08 15:27:39.692690553 +0000 UTC m=+1107.778640587 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs") pod "autores-model-3bc9dc-kserve-5b7d748bc5-t4drf" (UID: "2546172d-8096-4569-a43f-cffe19560fa0") : secret "autores-model-3bc9dc-kserve-self-signed-certs" not found
Sep 08 15:27:35.694152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.694132    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b16d3a4c-5d48-479e-9cbe-53852ef8b35a" (UID: "b16d3a4c-5d48-479e-9cbe-53852ef8b35a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:27:35.694449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.694428    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b16d3a4c-5d48-479e-9cbe-53852ef8b35a" (UID: "b16d3a4c-5d48-479e-9cbe-53852ef8b35a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:27:35.694449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.694439    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-kube-api-access-rsw7j" (OuterVolumeSpecName: "kube-api-access-rsw7j") pod "b16d3a4c-5d48-479e-9cbe-53852ef8b35a" (UID: "b16d3a4c-5d48-479e-9cbe-53852ef8b35a"). InnerVolumeSpecName "kube-api-access-rsw7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:27:35.725754 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.725727    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:35.794613 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.794570    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-openshift-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:35.794613 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.794602    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-maas-api-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:35.794613 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.794615    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-metrics-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:35.794909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.794627    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsw7j\" (UniqueName: \"kubernetes.io/projected/b16d3a4c-5d48-479e-9cbe-53852ef8b35a-kube-api-access-rsw7j\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:35.833185 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.833098    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"]
Sep 08 15:27:35.833384 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.833361    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg" podUID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerName="istio-proxy" containerID="cri-o://eeb22375c7fc021494ad7a3bf03cb9368c1c3d099ac1eb93caeff632ac52c47d" gracePeriod=30
Sep 08 15:27:35.843748 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.843716    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:35.952934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:35.952894    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm" podUID="0aad21b6-b2cd-43fe-a741-226552691d82" containerName="main" containerID="cri-o://38a3f37b21f126481df8908d8bb2f7a6c71ef86adc34b80b9cfe8d8d55e21610" gracePeriod=45
Sep 08 15:27:36.099410 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.099328    2575 generic.go:356] "Generic (PLEG): container finished" podID="0aad21b6-b2cd-43fe-a741-226552691d82" containerID="38a3f37b21f126481df8908d8bb2f7a6c71ef86adc34b80b9cfe8d8d55e21610" exitCode=0
Sep 08 15:27:36.099573 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.099400    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm" event={"ID":"0aad21b6-b2cd-43fe-a741-226552691d82","Type":"ContainerDied","Data":"38a3f37b21f126481df8908d8bb2f7a6c71ef86adc34b80b9cfe8d8d55e21610"}
Sep 08 15:27:36.100904 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.100878    2575 generic.go:356] "Generic (PLEG): container finished" podID="b16d3a4c-5d48-479e-9cbe-53852ef8b35a" containerID="711462de2006f6f6376f1b18810f0f110ce5e16005f3ab6050be5fd6eb4c2413" exitCode=0
Sep 08 15:27:36.101015 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.100935    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"
Sep 08 15:27:36.101015 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.100948    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt" event={"ID":"b16d3a4c-5d48-479e-9cbe-53852ef8b35a","Type":"ContainerDied","Data":"711462de2006f6f6376f1b18810f0f110ce5e16005f3ab6050be5fd6eb4c2413"}
Sep 08 15:27:36.101015 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.101001    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt" event={"ID":"b16d3a4c-5d48-479e-9cbe-53852ef8b35a","Type":"ContainerDied","Data":"39977bac0669e496b8312dae36954deb9a43bd9a44094030119f23fdf67e5c52"}
Sep 08 15:27:36.101143 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.101031    2575 scope.go:117] "RemoveContainer" containerID="711462de2006f6f6376f1b18810f0f110ce5e16005f3ab6050be5fd6eb4c2413"
Sep 08 15:27:36.110659 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.110641    2575 scope.go:117] "RemoveContainer" containerID="711462de2006f6f6376f1b18810f0f110ce5e16005f3ab6050be5fd6eb4c2413"
Sep 08 15:27:36.110920 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:36.110902    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"711462de2006f6f6376f1b18810f0f110ce5e16005f3ab6050be5fd6eb4c2413\": container with ID starting with 711462de2006f6f6376f1b18810f0f110ce5e16005f3ab6050be5fd6eb4c2413 not found: ID does not exist" containerID="711462de2006f6f6376f1b18810f0f110ce5e16005f3ab6050be5fd6eb4c2413"
Sep 08 15:27:36.110995 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.110924    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711462de2006f6f6376f1b18810f0f110ce5e16005f3ab6050be5fd6eb4c2413"} err="failed to get container status \"711462de2006f6f6376f1b18810f0f110ce5e16005f3ab6050be5fd6eb4c2413\": rpc error: code = NotFound desc = could not find container \"711462de2006f6f6376f1b18810f0f110ce5e16005f3ab6050be5fd6eb4c2413\": container with ID starting with 711462de2006f6f6376f1b18810f0f110ce5e16005f3ab6050be5fd6eb4c2413 not found: ID does not exist"
Sep 08 15:27:36.123252 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.123231    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"]
Sep 08 15:27:36.131461 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.131440    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d1aa3ad8-7c66684fb9-hftlt"]
Sep 08 15:27:36.187270 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.187253    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:27:36.299822 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.299800    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-tmp-dir\") pod \"0aad21b6-b2cd-43fe-a741-226552691d82\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") "
Sep 08 15:27:36.299934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.299838    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-model-cache\") pod \"0aad21b6-b2cd-43fe-a741-226552691d82\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") "
Sep 08 15:27:36.299934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.299904    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-dshm\") pod \"0aad21b6-b2cd-43fe-a741-226552691d82\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") "
Sep 08 15:27:36.299934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.299921    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aad21b6-b2cd-43fe-a741-226552691d82-tls-certs\") pod \"0aad21b6-b2cd-43fe-a741-226552691d82\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") "
Sep 08 15:27:36.300085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.299945    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-home\") pod \"0aad21b6-b2cd-43fe-a741-226552691d82\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") "
Sep 08 15:27:36.300085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.299975    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6276r\" (UniqueName: \"kubernetes.io/projected/0aad21b6-b2cd-43fe-a741-226552691d82-kube-api-access-6276r\") pod \"0aad21b6-b2cd-43fe-a741-226552691d82\" (UID: \"0aad21b6-b2cd-43fe-a741-226552691d82\") "
Sep 08 15:27:36.300206 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.300067    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0aad21b6-b2cd-43fe-a741-226552691d82" (UID: "0aad21b6-b2cd-43fe-a741-226552691d82"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:36.300206 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.300170    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-model-cache" (OuterVolumeSpecName: "model-cache") pod "0aad21b6-b2cd-43fe-a741-226552691d82" (UID: "0aad21b6-b2cd-43fe-a741-226552691d82"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:36.300315 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.300201    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-home" (OuterVolumeSpecName: "home") pod "0aad21b6-b2cd-43fe-a741-226552691d82" (UID: "0aad21b6-b2cd-43fe-a741-226552691d82"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:36.300400 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.300385    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:36.300455 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.300401    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:36.300455 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.300410    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:36.301952 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.301933    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-dshm" (OuterVolumeSpecName: "dshm") pod "0aad21b6-b2cd-43fe-a741-226552691d82" (UID: "0aad21b6-b2cd-43fe-a741-226552691d82"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:36.302063 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.302024    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aad21b6-b2cd-43fe-a741-226552691d82-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0aad21b6-b2cd-43fe-a741-226552691d82" (UID: "0aad21b6-b2cd-43fe-a741-226552691d82"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:27:36.302063 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.302041    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aad21b6-b2cd-43fe-a741-226552691d82-kube-api-access-6276r" (OuterVolumeSpecName: "kube-api-access-6276r") pod "0aad21b6-b2cd-43fe-a741-226552691d82" (UID: "0aad21b6-b2cd-43fe-a741-226552691d82"). InnerVolumeSpecName "kube-api-access-6276r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:27:36.401725 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.401692    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6276r\" (UniqueName: \"kubernetes.io/projected/0aad21b6-b2cd-43fe-a741-226552691d82-kube-api-access-6276r\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:36.401725 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.401720    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aad21b6-b2cd-43fe-a741-226552691d82-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:36.401725 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.401729    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aad21b6-b2cd-43fe-a741-226552691d82-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:36.439083 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:36.439049    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b16d3a4c-5d48-479e-9cbe-53852ef8b35a" path="/var/lib/kubelet/pods/b16d3a4c-5d48-479e-9cbe-53852ef8b35a/volumes"
Sep 08 15:27:37.108115 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:37.108086    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"
Sep 08 15:27:37.108115 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:37.108102    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm" event={"ID":"0aad21b6-b2cd-43fe-a741-226552691d82","Type":"ContainerDied","Data":"f99ebfed92a323a3e9b338a870d0dda926f978092d1fb0b74e97eb3f7f6c26f3"}
Sep 08 15:27:37.108662 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:37.108160    2575 scope.go:117] "RemoveContainer" containerID="38a3f37b21f126481df8908d8bb2f7a6c71ef86adc34b80b9cfe8d8d55e21610"
Sep 08 15:27:37.127045 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:37.127017    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"]
Sep 08 15:27:37.131913 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:37.131888    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-af09fb66-kserve-648db48f9f-n72wm"]
Sep 08 15:27:37.740817 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:37.740782    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:37.856429 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:37.856394    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:38.431255 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:38.431225    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aad21b6-b2cd-43fe-a741-226552691d82" path="/var/lib/kubelet/pods/0aad21b6-b2cd-43fe-a741-226552691d82/volumes"
Sep 08 15:27:39.072660 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:39.072621    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:39.174270 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:39.174241    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:39.731885 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:39.731857    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-self-signed-certs: secret "autores-model-3bc9dc-kserve-self-signed-certs" not found
Sep 08 15:27:39.732243 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:39.731919    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs podName:2546172d-8096-4569-a43f-cffe19560fa0 nodeName:}" failed. No retries permitted until 2026-09-08 15:27:47.7319051 +0000 UTC m=+1115.817855119 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs") pod "autores-model-3bc9dc-kserve-5b7d748bc5-t4drf" (UID: "2546172d-8096-4569-a43f-cffe19560fa0") : secret "autores-model-3bc9dc-kserve-self-signed-certs" not found
Sep 08 15:27:39.942324 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:39.942297    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:39.942471 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:39.942372    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:39.953977 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:39.953964    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:40.136638 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.136618    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:27:40.249684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.249655    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"]
Sep 08 15:27:40.250246 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.250226    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b16d3a4c-5d48-479e-9cbe-53852ef8b35a" containerName="maas-api"
Sep 08 15:27:40.250294 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.250251    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b16d3a4c-5d48-479e-9cbe-53852ef8b35a" containerName="maas-api"
Sep 08 15:27:40.250332 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.250297    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aad21b6-b2cd-43fe-a741-226552691d82" containerName="main"
Sep 08 15:27:40.250332 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.250307    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aad21b6-b2cd-43fe-a741-226552691d82" containerName="main"
Sep 08 15:27:40.250423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.250411    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b16d3a4c-5d48-479e-9cbe-53852ef8b35a" containerName="maas-api"
Sep 08 15:27:40.250465 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.250430    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aad21b6-b2cd-43fe-a741-226552691d82" containerName="main"
Sep 08 15:27:40.255576 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.255554    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:40.260443 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.260424    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9e22317f-serving-cert\""
Sep 08 15:27:40.270285 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.270263    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"]
Sep 08 15:27:40.309852 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.309827    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:40.336332 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.336311    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-maas-api-tls\") pod \"maas-api-e2e-ait-9e22317f-6f76968d88-flpdm\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:40.336435 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.336346    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c3b1949-70bc-43fe-b92a-cfd123b266ce-openshift-service-ca\") pod \"maas-api-e2e-ait-9e22317f-6f76968d88-flpdm\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:40.336520 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.336481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxksl\" (UniqueName: \"kubernetes.io/projected/9c3b1949-70bc-43fe-b92a-cfd123b266ce-kube-api-access-vxksl\") pod \"maas-api-e2e-ait-9e22317f-6f76968d88-flpdm\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:40.336613 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.336592    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-metrics-certs\") pod \"maas-api-e2e-ait-9e22317f-6f76968d88-flpdm\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:40.408156 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.408068    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:40.437275 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.437254    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxksl\" (UniqueName: \"kubernetes.io/projected/9c3b1949-70bc-43fe-b92a-cfd123b266ce-kube-api-access-vxksl\") pod \"maas-api-e2e-ait-9e22317f-6f76968d88-flpdm\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:40.437394 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.437307    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-metrics-certs\") pod \"maas-api-e2e-ait-9e22317f-6f76968d88-flpdm\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:40.437394 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.437340    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-maas-api-tls\") pod \"maas-api-e2e-ait-9e22317f-6f76968d88-flpdm\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:40.437394 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.437360    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c3b1949-70bc-43fe-b92a-cfd123b266ce-openshift-service-ca\") pod \"maas-api-e2e-ait-9e22317f-6f76968d88-flpdm\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:40.437584 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:40.437487    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-serving-cert: secret "maas-api-e2e-ait-9e22317f-serving-cert" not found
Sep 08 15:27:40.437584 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:40.437578    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-maas-api-tls podName:9c3b1949-70bc-43fe-b92a-cfd123b266ce nodeName:}" failed. No retries permitted until 2026-09-08 15:27:40.937563517 +0000 UTC m=+1109.023513537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-maas-api-tls") pod "maas-api-e2e-ait-9e22317f-6f76968d88-flpdm" (UID: "9c3b1949-70bc-43fe-b92a-cfd123b266ce") : secret "maas-api-e2e-ait-9e22317f-serving-cert" not found
Sep 08 15:27:40.439792 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.439774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c3b1949-70bc-43fe-b92a-cfd123b266ce-openshift-service-ca\") pod \"maas-api-e2e-ait-9e22317f-6f76968d88-flpdm\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:40.440387 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.440370    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-metrics-certs\") pod \"maas-api-e2e-ait-9e22317f-6f76968d88-flpdm\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:40.445771 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.445752    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxksl\" (UniqueName: \"kubernetes.io/projected/9c3b1949-70bc-43fe-b92a-cfd123b266ce-kube-api-access-vxksl\") pod \"maas-api-e2e-ait-9e22317f-6f76968d88-flpdm\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:40.600544 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.600513    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:40.763109 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.763039    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:40.857937 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.857907    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:40.942426 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.942401    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-maas-api-tls\") pod \"maas-api-e2e-ait-9e22317f-6f76968d88-flpdm\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:40.944688 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:40.944666    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-maas-api-tls\") pod \"maas-api-e2e-ait-9e22317f-6f76968d88-flpdm\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:41.071125 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.071098    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:27:41.132361 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.132336    2575 generic.go:356] "Generic (PLEG): container finished" podID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerID="eeb22375c7fc021494ad7a3bf03cb9368c1c3d099ac1eb93caeff632ac52c47d" exitCode=0
Sep 08 15:27:41.132525 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.132386    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"
Sep 08 15:27:41.132525 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.132408    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg" event={"ID":"dd28511e-c0f8-4e14-9828-5cb5d8bdf537","Type":"ContainerDied","Data":"eeb22375c7fc021494ad7a3bf03cb9368c1c3d099ac1eb93caeff632ac52c47d"}
Sep 08 15:27:41.132525 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.132457    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg" event={"ID":"dd28511e-c0f8-4e14-9828-5cb5d8bdf537","Type":"ContainerDied","Data":"06310fcf2c6da80aa183f2307397daf96e021055d7dd8101446c99bc9674157a"}
Sep 08 15:27:41.132525 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.132479    2575 scope.go:117] "RemoveContainer" containerID="eeb22375c7fc021494ad7a3bf03cb9368c1c3d099ac1eb93caeff632ac52c47d"
Sep 08 15:27:41.140776 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.140753    2575 scope.go:117] "RemoveContainer" containerID="8d279e8eb959692e1df5fc29766eac1b3f7c0c5bdfce513a7f6815e244a8a84e"
Sep 08 15:27:41.143900 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.143878    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istiod-ca-cert\") pod \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") "
Sep 08 15:27:41.144021 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144008    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-data\") pod \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") "
Sep 08 15:27:41.144084 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144041    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7gkl\" (UniqueName: \"kubernetes.io/projected/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-kube-api-access-n7gkl\") pod \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") "
Sep 08 15:27:41.144143 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144093    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-workload-socket\") pod \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") "
Sep 08 15:27:41.144143 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144120    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-workload-certs\") pod \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") "
Sep 08 15:27:41.144245 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144148    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-envoy\") pod \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") "
Sep 08 15:27:41.144245 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144177    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-credential-socket\") pod \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") "
Sep 08 15:27:41.144245 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144207    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-token\") pod \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") "
Sep 08 15:27:41.144245 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144231    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-podinfo\") pod \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\" (UID: \"dd28511e-c0f8-4e14-9828-5cb5d8bdf537\") "
Sep 08 15:27:41.144426 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144212    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd28511e-c0f8-4e14-9828-5cb5d8bdf537" (UID: "dd28511e-c0f8-4e14-9828-5cb5d8bdf537"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:27:41.144426 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144296    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd28511e-c0f8-4e14-9828-5cb5d8bdf537" (UID: "dd28511e-c0f8-4e14-9828-5cb5d8bdf537"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:41.144426 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144368    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd28511e-c0f8-4e14-9828-5cb5d8bdf537" (UID: "dd28511e-c0f8-4e14-9828-5cb5d8bdf537"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:41.144591 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144429    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd28511e-c0f8-4e14-9828-5cb5d8bdf537" (UID: "dd28511e-c0f8-4e14-9828-5cb5d8bdf537"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:41.144650 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144622    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd28511e-c0f8-4e14-9828-5cb5d8bdf537" (UID: "dd28511e-c0f8-4e14-9828-5cb5d8bdf537"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:41.144737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144720    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:41.144795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144738    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:41.144795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144748    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:41.144795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144762    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:41.144795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.144774    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:41.146306 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.146281    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-kube-api-access-n7gkl" (OuterVolumeSpecName: "kube-api-access-n7gkl") pod "dd28511e-c0f8-4e14-9828-5cb5d8bdf537" (UID: "dd28511e-c0f8-4e14-9828-5cb5d8bdf537"). InnerVolumeSpecName "kube-api-access-n7gkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:27:41.146383 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.146357    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd28511e-c0f8-4e14-9828-5cb5d8bdf537" (UID: "dd28511e-c0f8-4e14-9828-5cb5d8bdf537"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:27:41.146647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.146622    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd28511e-c0f8-4e14-9828-5cb5d8bdf537" (UID: "dd28511e-c0f8-4e14-9828-5cb5d8bdf537"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:41.146733 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.146622    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd28511e-c0f8-4e14-9828-5cb5d8bdf537" (UID: "dd28511e-c0f8-4e14-9828-5cb5d8bdf537"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:27:41.149070 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.149046    2575 scope.go:117] "RemoveContainer" containerID="eeb22375c7fc021494ad7a3bf03cb9368c1c3d099ac1eb93caeff632ac52c47d"
Sep 08 15:27:41.149312 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:41.149295    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeb22375c7fc021494ad7a3bf03cb9368c1c3d099ac1eb93caeff632ac52c47d\": container with ID starting with eeb22375c7fc021494ad7a3bf03cb9368c1c3d099ac1eb93caeff632ac52c47d not found: ID does not exist" containerID="eeb22375c7fc021494ad7a3bf03cb9368c1c3d099ac1eb93caeff632ac52c47d"
Sep 08 15:27:41.149355 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.149326    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeb22375c7fc021494ad7a3bf03cb9368c1c3d099ac1eb93caeff632ac52c47d"} err="failed to get container status \"eeb22375c7fc021494ad7a3bf03cb9368c1c3d099ac1eb93caeff632ac52c47d\": rpc error: code = NotFound desc = could not find container \"eeb22375c7fc021494ad7a3bf03cb9368c1c3d099ac1eb93caeff632ac52c47d\": container with ID starting with eeb22375c7fc021494ad7a3bf03cb9368c1c3d099ac1eb93caeff632ac52c47d not found: ID does not exist"
Sep 08 15:27:41.149355 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.149344    2575 scope.go:117] "RemoveContainer" containerID="8d279e8eb959692e1df5fc29766eac1b3f7c0c5bdfce513a7f6815e244a8a84e"
Sep 08 15:27:41.149680 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:41.149663    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d279e8eb959692e1df5fc29766eac1b3f7c0c5bdfce513a7f6815e244a8a84e\": container with ID starting with 8d279e8eb959692e1df5fc29766eac1b3f7c0c5bdfce513a7f6815e244a8a84e not found: ID does not exist" containerID="8d279e8eb959692e1df5fc29766eac1b3f7c0c5bdfce513a7f6815e244a8a84e"
Sep 08 15:27:41.149722 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.149688    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d279e8eb959692e1df5fc29766eac1b3f7c0c5bdfce513a7f6815e244a8a84e"} err="failed to get container status \"8d279e8eb959692e1df5fc29766eac1b3f7c0c5bdfce513a7f6815e244a8a84e\": rpc error: code = NotFound desc = could not find container \"8d279e8eb959692e1df5fc29766eac1b3f7c0c5bdfce513a7f6815e244a8a84e\": container with ID starting with 8d279e8eb959692e1df5fc29766eac1b3f7c0c5bdfce513a7f6815e244a8a84e not found: ID does not exist"
Sep 08 15:27:41.166875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.166858    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:41.246684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.246654    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7gkl\" (UniqueName: \"kubernetes.io/projected/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-kube-api-access-n7gkl\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:41.246808 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.246689    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:41.246808 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.246703    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:41.246808 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.246715    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd28511e-c0f8-4e14-9828-5cb5d8bdf537-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:41.438613 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.438590    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"]
Sep 08 15:27:41.441038 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:27:41.441012    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c3b1949_70bc_43fe_b92a_cfd123b266ce.slice/crio-6bdf48fadb9c650134ed0e51579b84d71e71db092209b7eea1122de922a3f846 WatchSource:0}: Error finding container 6bdf48fadb9c650134ed0e51579b84d71e71db092209b7eea1122de922a3f846: Status 404 returned error can't find the container with id 6bdf48fadb9c650134ed0e51579b84d71e71db092209b7eea1122de922a3f846
Sep 08 15:27:41.470604 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.470583    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"]
Sep 08 15:27:41.474760 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:41.474741    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-d1aa3ad8-gw-openshift-default-6869456b68-8z7sg"]
Sep 08 15:27:42.031288 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.031259    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"]
Sep 08 15:27:42.031704 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.031512    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l" podUID="c7999d7e-8921-4542-ad1b-142cb114dc49" containerName="maas-api" containerID="cri-o://b1e47a9d4f471d15cad0caf68ce8dfc6d6d8fec8898848622dea82a3f7e26b9e" gracePeriod=30
Sep 08 15:27:42.067961 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.067936    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:42.141328 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.141295    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm" event={"ID":"9c3b1949-70bc-43fe-b92a-cfd123b266ce","Type":"ContainerStarted","Data":"6bdf48fadb9c650134ed0e51579b84d71e71db092209b7eea1122de922a3f846"}
Sep 08 15:27:42.169589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.169561    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:42.268755 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.268685    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:42.269948 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.269930    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:27:42.352288 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.352215    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:42.355261 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.355243    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-maas-api-tls\") pod \"c7999d7e-8921-4542-ad1b-142cb114dc49\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") "
Sep 08 15:27:42.355315 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.355276    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzf4p\" (UniqueName: \"kubernetes.io/projected/c7999d7e-8921-4542-ad1b-142cb114dc49-kube-api-access-xzf4p\") pod \"c7999d7e-8921-4542-ad1b-142cb114dc49\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") "
Sep 08 15:27:42.355315 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.355313    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-metrics-certs\") pod \"c7999d7e-8921-4542-ad1b-142cb114dc49\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") "
Sep 08 15:27:42.355380 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.355352    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7999d7e-8921-4542-ad1b-142cb114dc49-openshift-service-ca\") pod \"c7999d7e-8921-4542-ad1b-142cb114dc49\" (UID: \"c7999d7e-8921-4542-ad1b-142cb114dc49\") "
Sep 08 15:27:42.355795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.355768    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7999d7e-8921-4542-ad1b-142cb114dc49-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c7999d7e-8921-4542-ad1b-142cb114dc49" (UID: "c7999d7e-8921-4542-ad1b-142cb114dc49"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:27:42.357332 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.357309    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7999d7e-8921-4542-ad1b-142cb114dc49-kube-api-access-xzf4p" (OuterVolumeSpecName: "kube-api-access-xzf4p") pod "c7999d7e-8921-4542-ad1b-142cb114dc49" (UID: "c7999d7e-8921-4542-ad1b-142cb114dc49"). InnerVolumeSpecName "kube-api-access-xzf4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:27:42.357421 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.357344    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c7999d7e-8921-4542-ad1b-142cb114dc49" (UID: "c7999d7e-8921-4542-ad1b-142cb114dc49"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:27:42.357421 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.357363    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c7999d7e-8921-4542-ad1b-142cb114dc49" (UID: "c7999d7e-8921-4542-ad1b-142cb114dc49"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:27:42.433453 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.433422    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" path="/var/lib/kubelet/pods/dd28511e-c0f8-4e14-9828-5cb5d8bdf537/volumes"
Sep 08 15:27:42.456270 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.456247    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7999d7e-8921-4542-ad1b-142cb114dc49-openshift-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:42.456270 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.456271    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-maas-api-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:42.456404 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.456281    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzf4p\" (UniqueName: \"kubernetes.io/projected/c7999d7e-8921-4542-ad1b-142cb114dc49-kube-api-access-xzf4p\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:42.456404 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.456289    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7999d7e-8921-4542-ad1b-142cb114dc49-metrics-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:42.733561 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:42.733525    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:43.146948 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:43.146922    2575 generic.go:356] "Generic (PLEG): container finished" podID="c7999d7e-8921-4542-ad1b-142cb114dc49" containerID="b1e47a9d4f471d15cad0caf68ce8dfc6d6d8fec8898848622dea82a3f7e26b9e" exitCode=0
Sep 08 15:27:43.147400 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:43.146974    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"
Sep 08 15:27:43.147400 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:43.146999    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l" event={"ID":"c7999d7e-8921-4542-ad1b-142cb114dc49","Type":"ContainerDied","Data":"b1e47a9d4f471d15cad0caf68ce8dfc6d6d8fec8898848622dea82a3f7e26b9e"}
Sep 08 15:27:43.147400 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:43.147038    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l" event={"ID":"c7999d7e-8921-4542-ad1b-142cb114dc49","Type":"ContainerDied","Data":"7607de308bf87e6acff20f43dbf56788973bf79e89a313266e0f2e2745b944dc"}
Sep 08 15:27:43.147400 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:43.147059    2575 scope.go:117] "RemoveContainer" containerID="b1e47a9d4f471d15cad0caf68ce8dfc6d6d8fec8898848622dea82a3f7e26b9e"
Sep 08 15:27:43.155343 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:43.155322    2575 scope.go:117] "RemoveContainer" containerID="b1e47a9d4f471d15cad0caf68ce8dfc6d6d8fec8898848622dea82a3f7e26b9e"
Sep 08 15:27:43.155630 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:43.155610    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1e47a9d4f471d15cad0caf68ce8dfc6d6d8fec8898848622dea82a3f7e26b9e\": container with ID starting with b1e47a9d4f471d15cad0caf68ce8dfc6d6d8fec8898848622dea82a3f7e26b9e not found: ID does not exist" containerID="b1e47a9d4f471d15cad0caf68ce8dfc6d6d8fec8898848622dea82a3f7e26b9e"
Sep 08 15:27:43.155709 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:43.155633    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e47a9d4f471d15cad0caf68ce8dfc6d6d8fec8898848622dea82a3f7e26b9e"} err="failed to get container status \"b1e47a9d4f471d15cad0caf68ce8dfc6d6d8fec8898848622dea82a3f7e26b9e\": rpc error: code = NotFound desc = could not find container \"b1e47a9d4f471d15cad0caf68ce8dfc6d6d8fec8898848622dea82a3f7e26b9e\": container with ID starting with b1e47a9d4f471d15cad0caf68ce8dfc6d6d8fec8898848622dea82a3f7e26b9e not found: ID does not exist"
Sep 08 15:27:43.165296 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:43.165278    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"]
Sep 08 15:27:43.170357 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:43.170337    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-3bc9dc-9849958c6-v7s5l"]
Sep 08 15:27:44.154643 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:44.154611    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm" event={"ID":"9c3b1949-70bc-43fe-b92a-cfd123b266ce","Type":"ContainerStarted","Data":"49f357f012a9c3462d3a3676e47e52cc0ec94f0dceeac5c3ef59a62ff2088244"}
Sep 08 15:27:44.155066 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:44.154738    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:44.185986 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:44.185946    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm" podStartSLOduration=1.9934098420000002 podStartE2EDuration="4.185932778s" totalImagesPullingTime="2.192522936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:27:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:27:44.185737178 +0000 UTC m=+1112.271687220" watchObservedRunningTime="2026-09-08 15:27:44.185932778 +0000 UTC m=+1112.271882818"
Sep 08 15:27:44.433394 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:44.433325    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7999d7e-8921-4542-ad1b-142cb114dc49" path="/var/lib/kubelet/pods/c7999d7e-8921-4542-ad1b-142cb114dc49/volumes"
Sep 08 15:27:47.810533 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:47.810480    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-self-signed-certs: secret "autores-model-3bc9dc-kserve-self-signed-certs" not found
Sep 08 15:27:47.810969 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:47.810574    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs podName:2546172d-8096-4569-a43f-cffe19560fa0 nodeName:}" failed. No retries permitted until 2026-09-08 15:28:03.810560048 +0000 UTC m=+1131.896510067 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs") pod "autores-model-3bc9dc-kserve-5b7d748bc5-t4drf" (UID: "2546172d-8096-4569-a43f-cffe19560fa0") : secret "autores-model-3bc9dc-kserve-self-signed-certs" not found
Sep 08 15:27:47.816987 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:47.816956    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:48.099730 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.099640    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf" podUID="2546172d-8096-4569-a43f-cffe19560fa0" containerName="main" containerID="cri-o://67e2314e4ede69ec183e7f022294101506b692e586f29937d0cbf3fa6df425fb" gracePeriod=45
Sep 08 15:27:48.366441 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.366417    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:48.516976 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.516948    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-tmp-dir\") pod \"2546172d-8096-4569-a43f-cffe19560fa0\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") "
Sep 08 15:27:48.517152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.517001    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs\") pod \"2546172d-8096-4569-a43f-cffe19560fa0\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") "
Sep 08 15:27:48.517152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.517041    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czwjw\" (UniqueName: \"kubernetes.io/projected/2546172d-8096-4569-a43f-cffe19560fa0-kube-api-access-czwjw\") pod \"2546172d-8096-4569-a43f-cffe19560fa0\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") "
Sep 08 15:27:48.517275 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.517157    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-model-cache\") pod \"2546172d-8096-4569-a43f-cffe19560fa0\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") "
Sep 08 15:27:48.517275 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.517218    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-dshm\") pod \"2546172d-8096-4569-a43f-cffe19560fa0\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") "
Sep 08 15:27:48.517275 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.517228    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2546172d-8096-4569-a43f-cffe19560fa0" (UID: "2546172d-8096-4569-a43f-cffe19560fa0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:48.517275 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.517245    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-home\") pod \"2546172d-8096-4569-a43f-cffe19560fa0\" (UID: \"2546172d-8096-4569-a43f-cffe19560fa0\") "
Sep 08 15:27:48.517518 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.517396    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-model-cache" (OuterVolumeSpecName: "model-cache") pod "2546172d-8096-4569-a43f-cffe19560fa0" (UID: "2546172d-8096-4569-a43f-cffe19560fa0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:48.517591 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.517558    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-home" (OuterVolumeSpecName: "home") pod "2546172d-8096-4569-a43f-cffe19560fa0" (UID: "2546172d-8096-4569-a43f-cffe19560fa0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:48.517738 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.517675    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:48.517738 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.517696    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:48.517738 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.517708    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:48.519839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.519810    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-dshm" (OuterVolumeSpecName: "dshm") pod "2546172d-8096-4569-a43f-cffe19560fa0" (UID: "2546172d-8096-4569-a43f-cffe19560fa0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:48.519950 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.519844    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2546172d-8096-4569-a43f-cffe19560fa0-kube-api-access-czwjw" (OuterVolumeSpecName: "kube-api-access-czwjw") pod "2546172d-8096-4569-a43f-cffe19560fa0" (UID: "2546172d-8096-4569-a43f-cffe19560fa0"). InnerVolumeSpecName "kube-api-access-czwjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:27:48.519950 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.519886    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2546172d-8096-4569-a43f-cffe19560fa0" (UID: "2546172d-8096-4569-a43f-cffe19560fa0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:27:48.618730 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.618654    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2546172d-8096-4569-a43f-cffe19560fa0-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:48.618730 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.618686    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2546172d-8096-4569-a43f-cffe19560fa0-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:48.618730 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.618704    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czwjw\" (UniqueName: \"kubernetes.io/projected/2546172d-8096-4569-a43f-cffe19560fa0-kube-api-access-czwjw\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:48.657069 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.657040    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q" podUID="a1db9b37-f55a-4b92-848c-9427bfa957f6" containerName="main" containerID="cri-o://181a2a36530c1627c19131f78ac5fe46c3ced001864cba32b6ce55c8bc3c8f9e" gracePeriod=45
Sep 08 15:27:48.895019 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:48.894995    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:27:49.022752 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.022729    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-home\") pod \"a1db9b37-f55a-4b92-848c-9427bfa957f6\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") "
Sep 08 15:27:49.022880 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.022791    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7bhh\" (UniqueName: \"kubernetes.io/projected/a1db9b37-f55a-4b92-848c-9427bfa957f6-kube-api-access-f7bhh\") pod \"a1db9b37-f55a-4b92-848c-9427bfa957f6\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") "
Sep 08 15:27:49.022880 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.022820    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1db9b37-f55a-4b92-848c-9427bfa957f6-tls-certs\") pod \"a1db9b37-f55a-4b92-848c-9427bfa957f6\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") "
Sep 08 15:27:49.022880 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.022855    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-model-cache\") pod \"a1db9b37-f55a-4b92-848c-9427bfa957f6\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") "
Sep 08 15:27:49.022880 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.022878    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-tmp-dir\") pod \"a1db9b37-f55a-4b92-848c-9427bfa957f6\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") "
Sep 08 15:27:49.023096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.022904    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-dshm\") pod \"a1db9b37-f55a-4b92-848c-9427bfa957f6\" (UID: \"a1db9b37-f55a-4b92-848c-9427bfa957f6\") "
Sep 08 15:27:49.023096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.022967    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-home" (OuterVolumeSpecName: "home") pod "a1db9b37-f55a-4b92-848c-9427bfa957f6" (UID: "a1db9b37-f55a-4b92-848c-9427bfa957f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:49.023198 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.023099    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-model-cache" (OuterVolumeSpecName: "model-cache") pod "a1db9b37-f55a-4b92-848c-9427bfa957f6" (UID: "a1db9b37-f55a-4b92-848c-9427bfa957f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:49.023198 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.023139    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a1db9b37-f55a-4b92-848c-9427bfa957f6" (UID: "a1db9b37-f55a-4b92-848c-9427bfa957f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:49.023310 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.023296    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:49.023362 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.023314    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:49.023362 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.023324    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:49.024799 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.024774    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1db9b37-f55a-4b92-848c-9427bfa957f6-kube-api-access-f7bhh" (OuterVolumeSpecName: "kube-api-access-f7bhh") pod "a1db9b37-f55a-4b92-848c-9427bfa957f6" (UID: "a1db9b37-f55a-4b92-848c-9427bfa957f6"). InnerVolumeSpecName "kube-api-access-f7bhh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:27:49.024799 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.024784    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-dshm" (OuterVolumeSpecName: "dshm") pod "a1db9b37-f55a-4b92-848c-9427bfa957f6" (UID: "a1db9b37-f55a-4b92-848c-9427bfa957f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:27:49.024931 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.024780    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1db9b37-f55a-4b92-848c-9427bfa957f6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a1db9b37-f55a-4b92-848c-9427bfa957f6" (UID: "a1db9b37-f55a-4b92-848c-9427bfa957f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:27:49.124188 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.124164    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7bhh\" (UniqueName: \"kubernetes.io/projected/a1db9b37-f55a-4b92-848c-9427bfa957f6-kube-api-access-f7bhh\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:49.124188 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.124186    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1db9b37-f55a-4b92-848c-9427bfa957f6-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:49.124313 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.124196    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1db9b37-f55a-4b92-848c-9427bfa957f6-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:27:49.181130 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.181064    2575 generic.go:356] "Generic (PLEG): container finished" podID="a1db9b37-f55a-4b92-848c-9427bfa957f6" containerID="181a2a36530c1627c19131f78ac5fe46c3ced001864cba32b6ce55c8bc3c8f9e" exitCode=0
Sep 08 15:27:49.181130 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.181110    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"
Sep 08 15:27:49.181269 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.181154    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q" event={"ID":"a1db9b37-f55a-4b92-848c-9427bfa957f6","Type":"ContainerDied","Data":"181a2a36530c1627c19131f78ac5fe46c3ced001864cba32b6ce55c8bc3c8f9e"}
Sep 08 15:27:49.181269 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.181193    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q" event={"ID":"a1db9b37-f55a-4b92-848c-9427bfa957f6","Type":"ContainerDied","Data":"ef1cda87f8714d788afdc6fab4270bb69a35c74ffac7827f280548184643b0f8"}
Sep 08 15:27:49.181269 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.181213    2575 scope.go:117] "RemoveContainer" containerID="181a2a36530c1627c19131f78ac5fe46c3ced001864cba32b6ce55c8bc3c8f9e"
Sep 08 15:27:49.182616 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.182482    2575 generic.go:356] "Generic (PLEG): container finished" podID="2546172d-8096-4569-a43f-cffe19560fa0" containerID="67e2314e4ede69ec183e7f022294101506b692e586f29937d0cbf3fa6df425fb" exitCode=0
Sep 08 15:27:49.182616 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.182545    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"
Sep 08 15:27:49.182741 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.182551    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf" event={"ID":"2546172d-8096-4569-a43f-cffe19560fa0","Type":"ContainerDied","Data":"67e2314e4ede69ec183e7f022294101506b692e586f29937d0cbf3fa6df425fb"}
Sep 08 15:27:49.182741 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.182668    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf" event={"ID":"2546172d-8096-4569-a43f-cffe19560fa0","Type":"ContainerDied","Data":"a9a112c1d3eebca532bc0ee871c580ac4b412658fbd32dd1f329b0581f15aa77"}
Sep 08 15:27:49.190369 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.190275    2575 scope.go:117] "RemoveContainer" containerID="181a2a36530c1627c19131f78ac5fe46c3ced001864cba32b6ce55c8bc3c8f9e"
Sep 08 15:27:49.190739 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:49.190715    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"181a2a36530c1627c19131f78ac5fe46c3ced001864cba32b6ce55c8bc3c8f9e\": container with ID starting with 181a2a36530c1627c19131f78ac5fe46c3ced001864cba32b6ce55c8bc3c8f9e not found: ID does not exist" containerID="181a2a36530c1627c19131f78ac5fe46c3ced001864cba32b6ce55c8bc3c8f9e"
Sep 08 15:27:49.190833 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.190743    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"181a2a36530c1627c19131f78ac5fe46c3ced001864cba32b6ce55c8bc3c8f9e"} err="failed to get container status \"181a2a36530c1627c19131f78ac5fe46c3ced001864cba32b6ce55c8bc3c8f9e\": rpc error: code = NotFound desc = could not find container \"181a2a36530c1627c19131f78ac5fe46c3ced001864cba32b6ce55c8bc3c8f9e\": container with ID starting with 181a2a36530c1627c19131f78ac5fe46c3ced001864cba32b6ce55c8bc3c8f9e not found: ID does not exist"
Sep 08 15:27:49.190833 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.190759    2575 scope.go:117] "RemoveContainer" containerID="67e2314e4ede69ec183e7f022294101506b692e586f29937d0cbf3fa6df425fb"
Sep 08 15:27:49.198252 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.198234    2575 scope.go:117] "RemoveContainer" containerID="67e2314e4ede69ec183e7f022294101506b692e586f29937d0cbf3fa6df425fb"
Sep 08 15:27:49.198481 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:27:49.198464    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67e2314e4ede69ec183e7f022294101506b692e586f29937d0cbf3fa6df425fb\": container with ID starting with 67e2314e4ede69ec183e7f022294101506b692e586f29937d0cbf3fa6df425fb not found: ID does not exist" containerID="67e2314e4ede69ec183e7f022294101506b692e586f29937d0cbf3fa6df425fb"
Sep 08 15:27:49.198540 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.198501    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67e2314e4ede69ec183e7f022294101506b692e586f29937d0cbf3fa6df425fb"} err="failed to get container status \"67e2314e4ede69ec183e7f022294101506b692e586f29937d0cbf3fa6df425fb\": rpc error: code = NotFound desc = could not find container \"67e2314e4ede69ec183e7f022294101506b692e586f29937d0cbf3fa6df425fb\": container with ID starting with 67e2314e4ede69ec183e7f022294101506b692e586f29937d0cbf3fa6df425fb not found: ID does not exist"
Sep 08 15:27:49.204458 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.204440    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"]
Sep 08 15:27:49.207629 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.207611    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-af09fb66-kserve-5d8ccbd68f-v6h2q"]
Sep 08 15:27:49.217442 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.217424    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"]
Sep 08 15:27:49.220228 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:49.220208    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-3bc9dc/autores-model-3bc9dc-kserve-5b7d748bc5-t4drf"]
Sep 08 15:27:50.163990 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:50.163959    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:27:50.432419 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:50.432349    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2546172d-8096-4569-a43f-cffe19560fa0" path="/var/lib/kubelet/pods/2546172d-8096-4569-a43f-cffe19560fa0/volumes"
Sep 08 15:27:50.432736 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:50.432722    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1db9b37-f55a-4b92-848c-9427bfa957f6" path="/var/lib/kubelet/pods/a1db9b37-f55a-4b92-848c-9427bfa957f6/volumes"
Sep 08 15:27:54.482417 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:54.482385    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:54.703121 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:54.703085    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"]
Sep 08 15:27:54.703363 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:54.703334    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerName="istio-proxy" containerID="cri-o://3d203e22c6e509db550ea9bb7ba5501e1bf340f818b45ffe10df22307789571a" gracePeriod=30
Sep 08 15:27:54.727763 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:54.727736    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:55.187741 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.187710    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:55.467386 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467314    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"]
Sep 08 15:27:55.467722 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467709    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerName="istio-proxy"
Sep 08 15:27:55.467777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467724    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerName="istio-proxy"
Sep 08 15:27:55.467777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467738    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2546172d-8096-4569-a43f-cffe19560fa0" containerName="main"
Sep 08 15:27:55.467777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467747    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2546172d-8096-4569-a43f-cffe19560fa0" containerName="main"
Sep 08 15:27:55.467777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467765    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1db9b37-f55a-4b92-848c-9427bfa957f6" containerName="main"
Sep 08 15:27:55.467777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467770    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1db9b37-f55a-4b92-848c-9427bfa957f6" containerName="main"
Sep 08 15:27:55.467777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467775    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7999d7e-8921-4542-ad1b-142cb114dc49" containerName="maas-api"
Sep 08 15:27:55.467951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467780    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7999d7e-8921-4542-ad1b-142cb114dc49" containerName="maas-api"
Sep 08 15:27:55.467951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467855    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7999d7e-8921-4542-ad1b-142cb114dc49" containerName="maas-api"
Sep 08 15:27:55.467951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467863    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2546172d-8096-4569-a43f-cffe19560fa0" containerName="main"
Sep 08 15:27:55.467951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467869    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerName="istio-proxy"
Sep 08 15:27:55.467951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467876    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerName="istio-proxy"
Sep 08 15:27:55.467951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.467882    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1db9b37-f55a-4b92-848c-9427bfa957f6" containerName="main"
Sep 08 15:27:55.472580 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.472557    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.475300 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.475276    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-6de9d1-openshift-default-dockercfg-qfzrc\""
Sep 08 15:27:55.484479 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.484457    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"]
Sep 08 15:27:55.498879 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.498853    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:55.575350 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.575326    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-data\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.575458 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.575374    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdp28\" (UniqueName: \"kubernetes.io/projected/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-kube-api-access-kdp28\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.575550 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.575467    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-envoy\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.575590 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.575558    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-token\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.575590 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.575577    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-credential-socket\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.575674 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.575611    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-workload-certs\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.575674 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.575641    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-podinfo\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.575674 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.575658    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istiod-ca-cert\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.575781 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.575680    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-workload-socket\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.676351 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.676320    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-token\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.676541 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.676358    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-credential-socket\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.676541 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.676391    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-workload-certs\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.676541 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.676409    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-podinfo\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.676541 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.676430    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istiod-ca-cert\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.676541 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.676453    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-workload-socket\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.676541 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.676477    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-data\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.676541 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.676537    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdp28\" (UniqueName: \"kubernetes.io/projected/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-kube-api-access-kdp28\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.676897 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.676603    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-envoy\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.676897 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.676804    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-credential-socket\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.676897 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.676855    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-workload-certs\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.677052 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.676909    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-workload-socket\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.677052 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.676971    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-data\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.677183 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.677163    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istiod-ca-cert\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.678795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.678770    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-envoy\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.678982 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.678965    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-podinfo\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.685641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.685623    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdp28\" (UniqueName: \"kubernetes.io/projected/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-kube-api-access-kdp28\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.686182 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.686163    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-token\") pod \"e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") " pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:55.695188 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.695165    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:55.782706 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:55.782642    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:56.056546 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:56.056520    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"]
Sep 08 15:27:56.058280 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:27:56.058253    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ff4f19_3111_4fe9_934c_40e74d9b5eeb.slice/crio-0b0e6bc3ad4e908c11ae0d97578e65da546a7fd722c9b427102e2514592106b3 WatchSource:0}: Error finding container 0b0e6bc3ad4e908c11ae0d97578e65da546a7fd722c9b427102e2514592106b3: Status 404 returned error can't find the container with id 0b0e6bc3ad4e908c11ae0d97578e65da546a7fd722c9b427102e2514592106b3
Sep 08 15:27:56.060659 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:56.060623    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:27:56.060721 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:56.060705    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:27:56.060772 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:56.060750    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:27:56.223012 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:56.222972    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" event={"ID":"07ff4f19-3111-4fe9-934c-40e74d9b5eeb","Type":"ContainerStarted","Data":"a77bfc524ca3b30c923ab1efc9e26eb66e4762021808bbf84067b9cf8d876366"}
Sep 08 15:27:56.223187 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:56.223070    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" event={"ID":"07ff4f19-3111-4fe9-934c-40e74d9b5eeb","Type":"ContainerStarted","Data":"0b0e6bc3ad4e908c11ae0d97578e65da546a7fd722c9b427102e2514592106b3"}
Sep 08 15:27:56.243739 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:56.243685    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" podStartSLOduration=1.243669572 podStartE2EDuration="1.243669572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:27:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:27:56.2418315 +0000 UTC m=+1124.327781544" watchObservedRunningTime="2026-09-08 15:27:56.243669572 +0000 UTC m=+1124.329619614"
Sep 08 15:27:56.334576 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:56.334483    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:27:56.783307 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:56.783278    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:57.788400 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:57.788373    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:58.232894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:58.232862    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:58.233785 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:58.233763    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:27:59.195360 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.195325    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:59.266810 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.266781    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"]
Sep 08 15:27:59.267313 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.267292    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerName="istio-proxy"
Sep 08 15:27:59.267313 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.267314    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd28511e-c0f8-4e14-9828-5cb5d8bdf537" containerName="istio-proxy"
Sep 08 15:27:59.270817 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.270797    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.273215 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.273193    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-6de9d1-serving-cert\""
Sep 08 15:27:59.284965 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.284931    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"]
Sep 08 15:27:59.319241 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.319198    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:59.411594 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.411572    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2688761e-d688-496f-a17b-1b5afece2769-maas-api-tls\") pod \"maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.411889 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.411797    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2688761e-d688-496f-a17b-1b5afece2769-openshift-service-ca\") pod \"maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.411889 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.411891    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2688761e-d688-496f-a17b-1b5afece2769-metrics-certs\") pod \"maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.412531 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.411961    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wrcr\" (UniqueName: \"kubernetes.io/projected/2688761e-d688-496f-a17b-1b5afece2769-kube-api-access-6wrcr\") pod \"maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.418299 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.418266    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:27:59.513312 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.513253    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2688761e-d688-496f-a17b-1b5afece2769-maas-api-tls\") pod \"maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.513312 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.513296    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2688761e-d688-496f-a17b-1b5afece2769-openshift-service-ca\") pod \"maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.513456 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.513337    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2688761e-d688-496f-a17b-1b5afece2769-metrics-certs\") pod \"maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.513456 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.513368    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wrcr\" (UniqueName: \"kubernetes.io/projected/2688761e-d688-496f-a17b-1b5afece2769-kube-api-access-6wrcr\") pod \"maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.514077 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.514058    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2688761e-d688-496f-a17b-1b5afece2769-openshift-service-ca\") pod \"maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.515592 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.515571    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2688761e-d688-496f-a17b-1b5afece2769-maas-api-tls\") pod \"maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.515674 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.515633    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2688761e-d688-496f-a17b-1b5afece2769-metrics-certs\") pod \"maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.521164 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.521146    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wrcr\" (UniqueName: \"kubernetes.io/projected/2688761e-d688-496f-a17b-1b5afece2769-kube-api-access-6wrcr\") pod \"maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.581638 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.581621    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:27:59.857528 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.857501    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"]
Sep 08 15:27:59.858820 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:27:59.858798    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2688761e_d688_496f_a17b_1b5afece2769.slice/crio-264d5ee3b180b8f2a867822c5d851c28203c03c8114a82e1cc3d8df26dd3ceac WatchSource:0}: Error finding container 264d5ee3b180b8f2a867822c5d851c28203c03c8114a82e1cc3d8df26dd3ceac: Status 404 returned error can't find the container with id 264d5ee3b180b8f2a867822c5d851c28203c03c8114a82e1cc3d8df26dd3ceac
Sep 08 15:27:59.939380 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.939350    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:27:59.940067 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:27:59.940036    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:00.035272 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.035241    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:00.119254 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119223    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-workload-certs\") pod \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") "
Sep 08 15:28:00.119254 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119258    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-podinfo\") pod \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") "
Sep 08 15:28:00.119451 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119295    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wzrj\" (UniqueName: \"kubernetes.io/projected/c2f28e4b-890a-43c7-a1f2-152ec40553b2-kube-api-access-8wzrj\") pod \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") "
Sep 08 15:28:00.119451 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119311    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-data\") pod \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") "
Sep 08 15:28:00.119451 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119345    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-envoy\") pod \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") "
Sep 08 15:28:00.119451 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119379    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-credential-socket\") pod \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") "
Sep 08 15:28:00.119451 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119421    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-workload-socket\") pod \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") "
Sep 08 15:28:00.119451 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119441    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istiod-ca-cert\") pod \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") "
Sep 08 15:28:00.119775 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119517    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-token\") pod \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\" (UID: \"c2f28e4b-890a-43c7-a1f2-152ec40553b2\") "
Sep 08 15:28:00.119775 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119653    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c2f28e4b-890a-43c7-a1f2-152ec40553b2" (UID: "c2f28e4b-890a-43c7-a1f2-152ec40553b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:28:00.119775 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119676    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "c2f28e4b-890a-43c7-a1f2-152ec40553b2" (UID: "c2f28e4b-890a-43c7-a1f2-152ec40553b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:28:00.119921 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119834    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c2f28e4b-890a-43c7-a1f2-152ec40553b2" (UID: "c2f28e4b-890a-43c7-a1f2-152ec40553b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:28:00.119975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119934    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:00.119975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119954    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:00.119975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119953    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c2f28e4b-890a-43c7-a1f2-152ec40553b2" (UID: "c2f28e4b-890a-43c7-a1f2-152ec40553b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:28:00.119975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.119967    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:00.120166 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.120139    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c2f28e4b-890a-43c7-a1f2-152ec40553b2" (UID: "c2f28e4b-890a-43c7-a1f2-152ec40553b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:28:00.121519 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.121472    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c2f28e4b-890a-43c7-a1f2-152ec40553b2" (UID: "c2f28e4b-890a-43c7-a1f2-152ec40553b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:28:00.121686 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.121667    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "c2f28e4b-890a-43c7-a1f2-152ec40553b2" (UID: "c2f28e4b-890a-43c7-a1f2-152ec40553b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:28:00.121868 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.121852    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2f28e4b-890a-43c7-a1f2-152ec40553b2-kube-api-access-8wzrj" (OuterVolumeSpecName: "kube-api-access-8wzrj") pod "c2f28e4b-890a-43c7-a1f2-152ec40553b2" (UID: "c2f28e4b-890a-43c7-a1f2-152ec40553b2"). InnerVolumeSpecName "kube-api-access-8wzrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:28:00.121915 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.121898    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c2f28e4b-890a-43c7-a1f2-152ec40553b2" (UID: "c2f28e4b-890a-43c7-a1f2-152ec40553b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:28:00.220724 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.220701    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:00.220724 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.220722    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wzrj\" (UniqueName: \"kubernetes.io/projected/c2f28e4b-890a-43c7-a1f2-152ec40553b2-kube-api-access-8wzrj\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:00.221038 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.220733    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:00.221038 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.220742    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:00.221038 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.220749    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2f28e4b-890a-43c7-a1f2-152ec40553b2-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:00.221038 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.220757    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2f28e4b-890a-43c7-a1f2-152ec40553b2-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:00.244568 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.244544    2575 generic.go:356] "Generic (PLEG): container finished" podID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerID="3d203e22c6e509db550ea9bb7ba5501e1bf340f818b45ffe10df22307789571a" exitCode=0
Sep 08 15:28:00.244656 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.244617    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"
Sep 08 15:28:00.244713 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.244683    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" event={"ID":"c2f28e4b-890a-43c7-a1f2-152ec40553b2","Type":"ContainerDied","Data":"3d203e22c6e509db550ea9bb7ba5501e1bf340f818b45ffe10df22307789571a"}
Sep 08 15:28:00.244765 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.244736    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj" event={"ID":"c2f28e4b-890a-43c7-a1f2-152ec40553b2","Type":"ContainerDied","Data":"e93ef088b62b61edcf177ac5909788ce212b8ec0c83769045b80c6a61b3916aa"}
Sep 08 15:28:00.244816 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.244766    2575 scope.go:117] "RemoveContainer" containerID="3d203e22c6e509db550ea9bb7ba5501e1bf340f818b45ffe10df22307789571a"
Sep 08 15:28:00.250500 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.250475    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm" event={"ID":"2688761e-d688-496f-a17b-1b5afece2769","Type":"ContainerStarted","Data":"264d5ee3b180b8f2a867822c5d851c28203c03c8114a82e1cc3d8df26dd3ceac"}
Sep 08 15:28:00.256035 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.256019    2575 scope.go:117] "RemoveContainer" containerID="8696a126435a7925a47c3201dbc9e76036a771e740389a9720f1c811d4c1956e"
Sep 08 15:28:00.265018 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.264997    2575 scope.go:117] "RemoveContainer" containerID="3d203e22c6e509db550ea9bb7ba5501e1bf340f818b45ffe10df22307789571a"
Sep 08 15:28:00.265244 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:28:00.265228    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d203e22c6e509db550ea9bb7ba5501e1bf340f818b45ffe10df22307789571a\": container with ID starting with 3d203e22c6e509db550ea9bb7ba5501e1bf340f818b45ffe10df22307789571a not found: ID does not exist" containerID="3d203e22c6e509db550ea9bb7ba5501e1bf340f818b45ffe10df22307789571a"
Sep 08 15:28:00.265293 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.265252    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d203e22c6e509db550ea9bb7ba5501e1bf340f818b45ffe10df22307789571a"} err="failed to get container status \"3d203e22c6e509db550ea9bb7ba5501e1bf340f818b45ffe10df22307789571a\": rpc error: code = NotFound desc = could not find container \"3d203e22c6e509db550ea9bb7ba5501e1bf340f818b45ffe10df22307789571a\": container with ID starting with 3d203e22c6e509db550ea9bb7ba5501e1bf340f818b45ffe10df22307789571a not found: ID does not exist"
Sep 08 15:28:00.265293 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.265267    2575 scope.go:117] "RemoveContainer" containerID="8696a126435a7925a47c3201dbc9e76036a771e740389a9720f1c811d4c1956e"
Sep 08 15:28:00.265532 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:28:00.265517    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8696a126435a7925a47c3201dbc9e76036a771e740389a9720f1c811d4c1956e\": container with ID starting with 8696a126435a7925a47c3201dbc9e76036a771e740389a9720f1c811d4c1956e not found: ID does not exist" containerID="8696a126435a7925a47c3201dbc9e76036a771e740389a9720f1c811d4c1956e"
Sep 08 15:28:00.265579 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.265535    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8696a126435a7925a47c3201dbc9e76036a771e740389a9720f1c811d4c1956e"} err="failed to get container status \"8696a126435a7925a47c3201dbc9e76036a771e740389a9720f1c811d4c1956e\": rpc error: code = NotFound desc = could not find container \"8696a126435a7925a47c3201dbc9e76036a771e740389a9720f1c811d4c1956e\": container with ID starting with 8696a126435a7925a47c3201dbc9e76036a771e740389a9720f1c811d4c1956e not found: ID does not exist"
Sep 08 15:28:00.269160 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.269139    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"]
Sep 08 15:28:00.275269 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.275249    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-3bc9dc-openshift-default-7895fdb57d-fbzkj"]
Sep 08 15:28:00.431444 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.431380    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" path="/var/lib/kubelet/pods/c2f28e4b-890a-43c7-a1f2-152ec40553b2/volumes"
Sep 08 15:28:00.713341 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:00.713247    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:01.263673 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:01.263633    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm" event={"ID":"2688761e-d688-496f-a17b-1b5afece2769","Type":"ContainerStarted","Data":"1f51684d72f5582f636559f72ca9543bf16122ced23045987b8a6b4c8fc1f1df"}
Sep 08 15:28:01.264119 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:01.263727    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:28:01.282767 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:01.282724    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm" podStartSLOduration=1.9041554299999999 podStartE2EDuration="2.282710739s" totalImagesPullingTime="378.555309ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:28:01.280617868 +0000 UTC m=+1129.366567909" watchObservedRunningTime="2026-09-08 15:28:01.282710739 +0000 UTC m=+1129.368660779"
Sep 08 15:28:02.211750 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:02.211715    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:07.272722 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:07.272693    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:28:10.656538 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:10.656501    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"]
Sep 08 15:28:10.656965 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:10.656743    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm" podUID="9c3b1949-70bc-43fe-b92a-cfd123b266ce" containerName="maas-api" containerID="cri-o://49f357f012a9c3462d3a3676e47e52cc0ec94f0dceeac5c3ef59a62ff2088244" gracePeriod=30
Sep 08 15:28:10.683347 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:10.683317    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:10.786264 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:10.786230    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:10.901122 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:10.901098    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:28:10.946118 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:10.946057    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:11.007404 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.007376    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c3b1949-70bc-43fe-b92a-cfd123b266ce-openshift-service-ca\") pod \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") "
Sep 08 15:28:11.007518 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.007429    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-metrics-certs\") pod \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") "
Sep 08 15:28:11.007518 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.007485    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-maas-api-tls\") pod \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") "
Sep 08 15:28:11.007596 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.007546    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxksl\" (UniqueName: \"kubernetes.io/projected/9c3b1949-70bc-43fe-b92a-cfd123b266ce-kube-api-access-vxksl\") pod \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\" (UID: \"9c3b1949-70bc-43fe-b92a-cfd123b266ce\") "
Sep 08 15:28:11.007810 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.007777    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c3b1949-70bc-43fe-b92a-cfd123b266ce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9c3b1949-70bc-43fe-b92a-cfd123b266ce" (UID: "9c3b1949-70bc-43fe-b92a-cfd123b266ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:28:11.009554 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.009529    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9c3b1949-70bc-43fe-b92a-cfd123b266ce" (UID: "9c3b1949-70bc-43fe-b92a-cfd123b266ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:28:11.009643 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.009535    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c3b1949-70bc-43fe-b92a-cfd123b266ce-kube-api-access-vxksl" (OuterVolumeSpecName: "kube-api-access-vxksl") pod "9c3b1949-70bc-43fe-b92a-cfd123b266ce" (UID: "9c3b1949-70bc-43fe-b92a-cfd123b266ce"). InnerVolumeSpecName "kube-api-access-vxksl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:28:11.009643 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.009593    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9c3b1949-70bc-43fe-b92a-cfd123b266ce" (UID: "9c3b1949-70bc-43fe-b92a-cfd123b266ce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:28:11.021188 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.021142    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:11.109032 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.109004    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-maas-api-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:11.109032 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.109029    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxksl\" (UniqueName: \"kubernetes.io/projected/9c3b1949-70bc-43fe-b92a-cfd123b266ce-kube-api-access-vxksl\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:11.109228 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.109043    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c3b1949-70bc-43fe-b92a-cfd123b266ce-openshift-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:11.109228 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.109056    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c3b1949-70bc-43fe-b92a-cfd123b266ce-metrics-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:11.320390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.320303    2575 generic.go:356] "Generic (PLEG): container finished" podID="9c3b1949-70bc-43fe-b92a-cfd123b266ce" containerID="49f357f012a9c3462d3a3676e47e52cc0ec94f0dceeac5c3ef59a62ff2088244" exitCode=0
Sep 08 15:28:11.320390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.320352    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"
Sep 08 15:28:11.320603 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.320385    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm" event={"ID":"9c3b1949-70bc-43fe-b92a-cfd123b266ce","Type":"ContainerDied","Data":"49f357f012a9c3462d3a3676e47e52cc0ec94f0dceeac5c3ef59a62ff2088244"}
Sep 08 15:28:11.320603 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.320437    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm" event={"ID":"9c3b1949-70bc-43fe-b92a-cfd123b266ce","Type":"ContainerDied","Data":"6bdf48fadb9c650134ed0e51579b84d71e71db092209b7eea1122de922a3f846"}
Sep 08 15:28:11.320603 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.320460    2575 scope.go:117] "RemoveContainer" containerID="49f357f012a9c3462d3a3676e47e52cc0ec94f0dceeac5c3ef59a62ff2088244"
Sep 08 15:28:11.329797 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.329780    2575 scope.go:117] "RemoveContainer" containerID="49f357f012a9c3462d3a3676e47e52cc0ec94f0dceeac5c3ef59a62ff2088244"
Sep 08 15:28:11.330045 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:28:11.330029    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49f357f012a9c3462d3a3676e47e52cc0ec94f0dceeac5c3ef59a62ff2088244\": container with ID starting with 49f357f012a9c3462d3a3676e47e52cc0ec94f0dceeac5c3ef59a62ff2088244 not found: ID does not exist" containerID="49f357f012a9c3462d3a3676e47e52cc0ec94f0dceeac5c3ef59a62ff2088244"
Sep 08 15:28:11.330089 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.330054    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f357f012a9c3462d3a3676e47e52cc0ec94f0dceeac5c3ef59a62ff2088244"} err="failed to get container status \"49f357f012a9c3462d3a3676e47e52cc0ec94f0dceeac5c3ef59a62ff2088244\": rpc error: code = NotFound desc = could not find container \"49f357f012a9c3462d3a3676e47e52cc0ec94f0dceeac5c3ef59a62ff2088244\": container with ID starting with 49f357f012a9c3462d3a3676e47e52cc0ec94f0dceeac5c3ef59a62ff2088244 not found: ID does not exist"
Sep 08 15:28:11.344865 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.344841    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"]
Sep 08 15:28:11.348845 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.348825    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9e22317f-6f76968d88-flpdm"]
Sep 08 15:28:11.771551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:11.771522    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:12.109947 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.109920    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:12.437330 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.437251    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c3b1949-70bc-43fe-b92a-cfd123b266ce" path="/var/lib/kubelet/pods/9c3b1949-70bc-43fe-b92a-cfd123b266ce/volumes"
Sep 08 15:28:12.567694 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.567666    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:12.861973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.861942    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"]
Sep 08 15:28:12.862433 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.862416    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c3b1949-70bc-43fe-b92a-cfd123b266ce" containerName="maas-api"
Sep 08 15:28:12.862433 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.862433    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c3b1949-70bc-43fe-b92a-cfd123b266ce" containerName="maas-api"
Sep 08 15:28:12.862576 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.862457    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerName="istio-proxy"
Sep 08 15:28:12.862576 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.862463    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerName="istio-proxy"
Sep 08 15:28:12.862576 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.862471    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerName="istio-proxy"
Sep 08 15:28:12.862576 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.862476    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerName="istio-proxy"
Sep 08 15:28:12.862576 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.862561    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerName="istio-proxy"
Sep 08 15:28:12.862576 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.862568    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2f28e4b-890a-43c7-a1f2-152ec40553b2" containerName="istio-proxy"
Sep 08 15:28:12.862576 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.862574    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c3b1949-70bc-43fe-b92a-cfd123b266ce" containerName="maas-api"
Sep 08 15:28:12.867507 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.867476    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:12.870343 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.870321    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-152e35a7-openshift-default-dockercfg-sdn6n\""
Sep 08 15:28:12.878033 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.878010    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"]
Sep 08 15:28:12.902368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.902325    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:12.922717 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.922691    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istiod-ca-cert\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:12.922801 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.922720    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqmhg\" (UniqueName: \"kubernetes.io/projected/6b5a4117-fb0d-4727-9b1e-21c96127d43d-kube-api-access-wqmhg\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:12.922801 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.922779    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-data\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:12.922881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.922809    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-podinfo\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:12.922881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.922853    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-workload-certs\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:12.922881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.922877    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-credential-socket\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:12.922975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.922904    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-envoy\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:12.922975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.922953    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-workload-socket\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:12.923036 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:12.922969    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-token\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.024517 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.024419    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-podinfo\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.024517 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.024462    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-workload-certs\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.024517 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.024483    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-credential-socket\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.024781 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.024535    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-envoy\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.024781 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.024565    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-workload-socket\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.024781 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.024579    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-token\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.024781 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.024708    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istiod-ca-cert\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.024781 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.024747    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqmhg\" (UniqueName: \"kubernetes.io/projected/6b5a4117-fb0d-4727-9b1e-21c96127d43d-kube-api-access-wqmhg\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.025040 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.024796    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-data\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.025040 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.024854    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-workload-certs\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.025040 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.024937    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-credential-socket\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.025040 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.024959    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-workload-socket\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.025246 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.025151    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-data\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.025443 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.025424    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istiod-ca-cert\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.026631 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.026612    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-envoy\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.026837 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.026818    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-podinfo\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.030989 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.030961    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:13.032596 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.032575    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-token\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.033035 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.033019    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqmhg\" (UniqueName: \"kubernetes.io/projected/6b5a4117-fb0d-4727-9b1e-21c96127d43d-kube-api-access-wqmhg\") pod \"e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") " pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.178967 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.178940    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:13.453308 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.453281    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"]
Sep 08 15:28:13.455275 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:28:13.455253    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b5a4117_fb0d_4727_9b1e_21c96127d43d.slice/crio-8bd3086e3de8d37a6161204d5574b909eec4664efe859191ebc42b28c3f32703 WatchSource:0}: Error finding container 8bd3086e3de8d37a6161204d5574b909eec4664efe859191ebc42b28c3f32703: Status 404 returned error can't find the container with id 8bd3086e3de8d37a6161204d5574b909eec4664efe859191ebc42b28c3f32703
Sep 08 15:28:13.457444 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.457415    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:28:13.457530 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.457516    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:28:13.457574 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:13.457551    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:28:14.337344 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:14.337314    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r" event={"ID":"6b5a4117-fb0d-4727-9b1e-21c96127d43d","Type":"ContainerStarted","Data":"0403effebd631cbc049197837e4003ed2ff25d2dc48da5ecbf239e908485b518"}
Sep 08 15:28:14.337805 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:14.337373    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r" event={"ID":"6b5a4117-fb0d-4727-9b1e-21c96127d43d","Type":"ContainerStarted","Data":"8bd3086e3de8d37a6161204d5574b909eec4664efe859191ebc42b28c3f32703"}
Sep 08 15:28:14.363643 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:14.363597    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r" podStartSLOduration=2.3635834510000002 podStartE2EDuration="2.363583451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:28:14.362147425 +0000 UTC m=+1142.448097490" watchObservedRunningTime="2026-09-08 15:28:14.363583451 +0000 UTC m=+1142.449533491"
Sep 08 15:28:15.180022 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:15.179994    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:15.185077 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:15.185054    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:15.333753 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:15.333724    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:28:15.342418 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:15.342397    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:15.343328 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:15.343309    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:15.742609 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:15.742577    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:15.832057 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:15.832020    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:16.926645 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:16.926607    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"]
Sep 08 15:28:16.930246 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:16.930227    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:16.933091 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:16.933070    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-152e35a7-serving-cert\""
Sep 08 15:28:16.945192 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:16.945169    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"]
Sep 08 15:28:16.953162 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:16.953139    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:16.964314 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:16.964282    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-metrics-certs\") pod \"maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:16.964413 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:16.964358    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-openshift-service-ca\") pod \"maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:16.964559 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:16.964541    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-maas-api-tls\") pod \"maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:16.964601 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:16.964580    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9sg6\" (UniqueName: \"kubernetes.io/projected/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-kube-api-access-q9sg6\") pod \"maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:17.065195 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.065157    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-maas-api-tls\") pod \"maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:17.065195 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.065194    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9sg6\" (UniqueName: \"kubernetes.io/projected/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-kube-api-access-q9sg6\") pod \"maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:17.065394 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.065256    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-metrics-certs\") pod \"maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:17.065394 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.065289    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-openshift-service-ca\") pod \"maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:17.065394 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:28:17.065307    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-serving-cert: secret "maas-api-e2e-derive-152e35a7-serving-cert" not found
Sep 08 15:28:17.065394 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:28:17.065371    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-maas-api-tls podName:d8c2924e-19c4-46a2-ba2a-cfbfa5118719 nodeName:}" failed. No retries permitted until 2026-09-08 15:28:17.565351345 +0000 UTC m=+1145.651301388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-maas-api-tls") pod "maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4" (UID: "d8c2924e-19c4-46a2-ba2a-cfbfa5118719") : secret "maas-api-e2e-derive-152e35a7-serving-cert" not found
Sep 08 15:28:17.065950 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.065928    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-openshift-service-ca\") pod \"maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:17.067561 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.067540    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-metrics-certs\") pod \"maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:17.071934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.071904    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:17.080001 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.079974    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9sg6\" (UniqueName: \"kubernetes.io/projected/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-kube-api-access-q9sg6\") pod \"maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:17.208090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.207995    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:17.538156 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.538056    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:17.570030 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.570001    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-maas-api-tls\") pod \"maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:17.572291 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.572265    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-maas-api-tls\") pod \"maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:17.638519 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.638459    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:17.841179 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:17.841098    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:18.126972 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:18.126950    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"]
Sep 08 15:28:18.129022 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:28:18.128997    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8c2924e_19c4_46a2_ba2a_cfbfa5118719.slice/crio-645a809bce4c1841d972edeff2a59d430599af9b6541a8cabda943be1b8ab42d WatchSource:0}: Error finding container 645a809bce4c1841d972edeff2a59d430599af9b6541a8cabda943be1b8ab42d: Status 404 returned error can't find the container with id 645a809bce4c1841d972edeff2a59d430599af9b6541a8cabda943be1b8ab42d
Sep 08 15:28:18.357707 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:18.357676    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4" event={"ID":"d8c2924e-19c4-46a2-ba2a-cfbfa5118719","Type":"ContainerStarted","Data":"645a809bce4c1841d972edeff2a59d430599af9b6541a8cabda943be1b8ab42d"}
Sep 08 15:28:19.363807 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:19.363772    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4" event={"ID":"d8c2924e-19c4-46a2-ba2a-cfbfa5118719","Type":"ContainerStarted","Data":"3933ad44b043460495a01602b638c58b2397fef81da797ff1b5a5e9977cf93f5"}
Sep 08 15:28:19.364291 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:19.363893    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:19.386000 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:19.385957    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4" podStartSLOduration=2.850611798 podStartE2EDuration="3.385945582s" totalImagesPullingTime="535.333784ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:28:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:28:19.383738964 +0000 UTC m=+1147.469689004" watchObservedRunningTime="2026-09-08 15:28:19.385945582 +0000 UTC m=+1147.471895622"
Sep 08 15:28:20.396522 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:20.396462    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:23.183137 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:23.183101    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:25.375433 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:25.375406    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:25.399321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:25.399291    2575 generic.go:356] "Generic (PLEG): container finished" podID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerID="a77bfc524ca3b30c923ab1efc9e26eb66e4762021808bbf84067b9cf8d876366" exitCode=0
Sep 08 15:28:25.399518 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:25.399357    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" event={"ID":"07ff4f19-3111-4fe9-934c-40e74d9b5eeb","Type":"ContainerDied","Data":"a77bfc524ca3b30c923ab1efc9e26eb66e4762021808bbf84067b9cf8d876366"}
Sep 08 15:28:25.399748 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:25.399732    2575 scope.go:117] "RemoveContainer" containerID="a77bfc524ca3b30c923ab1efc9e26eb66e4762021808bbf84067b9cf8d876366"
Sep 08 15:28:25.401055 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:25.401018    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:28:25.401135 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:25.401102    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:28:25.401173 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:25.401149    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:28:26.410613 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:26.410562    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" event={"ID":"07ff4f19-3111-4fe9-934c-40e74d9b5eeb","Type":"ContainerStarted","Data":"fec5bb5450245911c6f2eac4d4eba79fcc32fa833334de8e2c605a69321abe45"}
Sep 08 15:28:26.783682 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:26.783601    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:28:26.784805 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:26.784775    2575 patch_prober.go:28] interesting pod/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 08 15:28:26.784953 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:26.784835    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 08 15:28:27.783732 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:27.783698    2575 patch_prober.go:28] interesting pod/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 08 15:28:27.784165 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:27.783753    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 08 15:28:28.233201 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:28.233157    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:28:28.783665 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:28.783628    2575 patch_prober.go:28] interesting pod/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 08 15:28:28.783855 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:28.783687    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 08 15:28:29.783964 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:29.783928    2575 patch_prober.go:28] interesting pod/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 08 15:28:29.784350 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:29.784008    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 08 15:28:30.783414 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:30.783375    2575 patch_prober.go:28] interesting pod/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 08 15:28:30.783615 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:30.783443    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 08 15:28:31.035999 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:31.035893    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:28:31.787155 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:31.787120    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:28:31.788254 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:31.788233    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:28:34.461817 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:34.461789    2575 generic.go:356] "Generic (PLEG): container finished" podID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerID="0403effebd631cbc049197837e4003ed2ff25d2dc48da5ecbf239e908485b518" exitCode=0
Sep 08 15:28:34.462183 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:34.461856    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r" event={"ID":"6b5a4117-fb0d-4727-9b1e-21c96127d43d","Type":"ContainerDied","Data":"0403effebd631cbc049197837e4003ed2ff25d2dc48da5ecbf239e908485b518"}
Sep 08 15:28:34.462252 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:34.462192    2575 scope.go:117] "RemoveContainer" containerID="0403effebd631cbc049197837e4003ed2ff25d2dc48da5ecbf239e908485b518"
Sep 08 15:28:34.463224 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:34.463192    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:28:34.463298 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:34.463284    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:28:34.463335 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:34.463317    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:28:35.468557 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:35.468523    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r" event={"ID":"6b5a4117-fb0d-4727-9b1e-21c96127d43d","Type":"ContainerStarted","Data":"336aab2dafa27a6625c09ee72e9fcde5085c6efeef24731673495db74f3aa03f"}
Sep 08 15:28:36.134364 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.134328    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"]
Sep 08 15:28:36.136928 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.136908    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.139747 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.139728    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-677a6c\"/\"openshift-service-ca.crt\""
Sep 08 15:28:36.141068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.141046    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-677a6c\"/\"test-model-677a6c-kserve-self-signed-certs\""
Sep 08 15:28:36.141187 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.141084    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-677a6c\"/\"default-dockercfg-c5q4v\""
Sep 08 15:28:36.141187 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.141119    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-677a6c\"/\"kube-root-ca.crt\""
Sep 08 15:28:36.147837 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.147813    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"]
Sep 08 15:28:36.179783 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.179762    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:36.179940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.179922    2575 patch_prober.go:28] interesting pod/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 08 15:28:36.179982 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.179961    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r" podUID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 08 15:28:36.241036 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.240997    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:36.265511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.264213    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-model-cache\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.265511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.264273    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-home\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.265511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.264313    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/358950a1-db0b-481f-92cb-ca5a31c53abd-tls-certs\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.265511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.264346    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-dshm\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.265511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.264371    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-tmp-dir\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.265511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.264415    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dkzz\" (UniqueName: \"kubernetes.io/projected/358950a1-db0b-481f-92cb-ca5a31c53abd-kube-api-access-4dkzz\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.365051 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.365019    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-home\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.365184 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.365061    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/358950a1-db0b-481f-92cb-ca5a31c53abd-tls-certs\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.365238 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.365182    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-dshm\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.365238 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.365219    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-tmp-dir\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.365309 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.365273    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dkzz\" (UniqueName: \"kubernetes.io/projected/358950a1-db0b-481f-92cb-ca5a31c53abd-kube-api-access-4dkzz\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.365423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.365405    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-home\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.365616 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.365597    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-tmp-dir\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.365696 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.365408    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-model-cache\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.365739 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.365703    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-model-cache\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.367352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.367336    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-dshm\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.367563 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.367544    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/358950a1-db0b-481f-92cb-ca5a31c53abd-tls-certs\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.374464 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.374443    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dkzz\" (UniqueName: \"kubernetes.io/projected/358950a1-db0b-481f-92cb-ca5a31c53abd-kube-api-access-4dkzz\") pod \"test-model-677a6c-kserve-5478b95d45-j8wb4\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") " pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.446198 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.446128    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:36.748127 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:36.748104    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"]
Sep 08 15:28:36.749709 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:28:36.749680    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod358950a1_db0b_481f_92cb_ca5a31c53abd.slice/crio-3ab784a38436a4eb412784bc3a66d7f6ac3373541739341dbd741e515dd2ce63 WatchSource:0}: Error finding container 3ab784a38436a4eb412784bc3a66d7f6ac3373541739341dbd741e515dd2ce63: Status 404 returned error can't find the container with id 3ab784a38436a4eb412784bc3a66d7f6ac3373541739341dbd741e515dd2ce63
Sep 08 15:28:37.180191 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:37.180158    2575 patch_prober.go:28] interesting pod/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 08 15:28:37.180366 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:37.180211    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r" podUID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 08 15:28:37.479955 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:37.479872    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4" event={"ID":"358950a1-db0b-481f-92cb-ca5a31c53abd","Type":"ContainerStarted","Data":"071a85523a8fa8d6e6b6d60b7dbe9574c6d3b576520fbc2edd8294a38a32f60e"}
Sep 08 15:28:37.479955 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:37.479908    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4" event={"ID":"358950a1-db0b-481f-92cb-ca5a31c53abd","Type":"ContainerStarted","Data":"3ab784a38436a4eb412784bc3a66d7f6ac3373541739341dbd741e515dd2ce63"}
Sep 08 15:28:37.949525 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:37.949470    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:38.180269 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:38.180236    2575 patch_prober.go:28] interesting pod/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 08 15:28:38.180424 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:38.180321    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r" podUID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 08 15:28:39.179725 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:39.179682    2575 patch_prober.go:28] interesting pod/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 08 15:28:39.180298 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:39.179749    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r" podUID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 08 15:28:40.180167 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:40.180134    2575 patch_prober.go:28] interesting pod/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 08 15:28:40.180634 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:40.180195    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r" podUID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 08 15:28:40.832340 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:40.832303    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:40.932636 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:40.932601    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:41.184009 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:41.183978    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:41.184384 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:41.184341    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:41.185072 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:41.185052    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:41.207321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:41.207258    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4" podStartSLOduration=5.207244952 podStartE2EDuration="5.207244952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:28:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:28:37.499986585 +0000 UTC m=+1165.585936627" watchObservedRunningTime="2026-09-08 15:28:41.207244952 +0000 UTC m=+1169.293195032"
Sep 08 15:28:46.446321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:46.446291    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:46.446321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:46.446328    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:46.458057 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:46.458021    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:46.534136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:46.534110    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:28:47.437544 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.437512    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"]
Sep 08 15:28:47.440914 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.440895    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.445082 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.445060    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-6de9d1\"/\"default-dockercfg-q4qz8\""
Sep 08 15:28:47.445209 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.445196    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-6de9d1\"/\"kube-root-ca.crt\""
Sep 08 15:28:47.445393 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.445146    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-6de9d1\"/\"openshift-service-ca.crt\""
Sep 08 15:28:47.445474 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.445424    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-6de9d1\"/\"explicit-model-6de9d1-kserve-self-signed-certs\""
Sep 08 15:28:47.455343 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.455322    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"]
Sep 08 15:28:47.466076 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.466055    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-model-cache\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.466178 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.466095    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69f24fab-d84d-4958-a99a-bc65fc857a18-tls-certs\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.466224 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.466196    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lk9x\" (UniqueName: \"kubernetes.io/projected/69f24fab-d84d-4958-a99a-bc65fc857a18-kube-api-access-4lk9x\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.466265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.466237    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-tmp-dir\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.466265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.466257    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-home\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.466337 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.466300    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-dshm\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.489037 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.489004    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:47.567102 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.567069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lk9x\" (UniqueName: \"kubernetes.io/projected/69f24fab-d84d-4958-a99a-bc65fc857a18-kube-api-access-4lk9x\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.567237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.567157    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-tmp-dir\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.567237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.567205    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-home\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.567355 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.567244    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-dshm\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.567412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.567353    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-model-cache\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.567469 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.567415    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69f24fab-d84d-4958-a99a-bc65fc857a18-tls-certs\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.567608 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.567589    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-tmp-dir\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.567672 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.567632    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-home\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.567890 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.567873    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-model-cache\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.569349 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.569321    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-dshm\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.569673 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.569658    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69f24fab-d84d-4958-a99a-bc65fc857a18-tls-certs\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.574977 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.574951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lk9x\" (UniqueName: \"kubernetes.io/projected/69f24fab-d84d-4958-a99a-bc65fc857a18-kube-api-access-4lk9x\") pod \"explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") " pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.625096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.625055    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"]
Sep 08 15:28:47.626179 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.626120    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4" podUID="d8c2924e-19c4-46a2-ba2a-cfbfa5118719" containerName="maas-api" containerID="cri-o://3933ad44b043460495a01602b638c58b2397fef81da797ff1b5a5e9977cf93f5" gracePeriod=30
Sep 08 15:28:47.664062 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.664036    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:47.752638 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.752607    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:47.860532 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.860502    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:47.971640 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.971609    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-maas-api-tls\") pod \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") "
Sep 08 15:28:47.971794 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.971695    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-openshift-service-ca\") pod \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") "
Sep 08 15:28:47.971794 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.971725    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-metrics-certs\") pod \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") "
Sep 08 15:28:47.971915 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.971791    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9sg6\" (UniqueName: \"kubernetes.io/projected/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-kube-api-access-q9sg6\") pod \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\" (UID: \"d8c2924e-19c4-46a2-ba2a-cfbfa5118719\") "
Sep 08 15:28:47.972793 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.972375    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d8c2924e-19c4-46a2-ba2a-cfbfa5118719" (UID: "d8c2924e-19c4-46a2-ba2a-cfbfa5118719"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:28:47.975738 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.975695    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-kube-api-access-q9sg6" (OuterVolumeSpecName: "kube-api-access-q9sg6") pod "d8c2924e-19c4-46a2-ba2a-cfbfa5118719" (UID: "d8c2924e-19c4-46a2-ba2a-cfbfa5118719"). InnerVolumeSpecName "kube-api-access-q9sg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:28:47.975840 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.975790    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d8c2924e-19c4-46a2-ba2a-cfbfa5118719" (UID: "d8c2924e-19c4-46a2-ba2a-cfbfa5118719"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:28:47.975840 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.975809    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d8c2924e-19c4-46a2-ba2a-cfbfa5118719" (UID: "d8c2924e-19c4-46a2-ba2a-cfbfa5118719"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:28:47.997598 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.997448    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"]
Sep 08 15:28:47.997761 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:47.997732    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r" podUID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerName="istio-proxy" containerID="cri-o://336aab2dafa27a6625c09ee72e9fcde5085c6efeef24731673495db74f3aa03f" gracePeriod=30
Sep 08 15:28:48.033345 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.033325    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"]
Sep 08 15:28:48.035398 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:28:48.035370    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69f24fab_d84d_4958_a99a_bc65fc857a18.slice/crio-05e7635d21049b9085f4c3397a6daf1a19d02f39142a0ca592678a09fee240eb WatchSource:0}: Error finding container 05e7635d21049b9085f4c3397a6daf1a19d02f39142a0ca592678a09fee240eb: Status 404 returned error can't find the container with id 05e7635d21049b9085f4c3397a6daf1a19d02f39142a0ca592678a09fee240eb
Sep 08 15:28:48.073181 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.073161    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9sg6\" (UniqueName: \"kubernetes.io/projected/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-kube-api-access-q9sg6\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:48.073269 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.073188    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-maas-api-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:48.073269 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.073202    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-openshift-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:48.073269 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.073213    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8c2924e-19c4-46a2-ba2a-cfbfa5118719-metrics-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:48.299101 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.299022    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:48.534749 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.534723    2575 generic.go:356] "Generic (PLEG): container finished" podID="d8c2924e-19c4-46a2-ba2a-cfbfa5118719" containerID="3933ad44b043460495a01602b638c58b2397fef81da797ff1b5a5e9977cf93f5" exitCode=0
Sep 08 15:28:48.535108 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.534766    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"
Sep 08 15:28:48.535108 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.534799    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4" event={"ID":"d8c2924e-19c4-46a2-ba2a-cfbfa5118719","Type":"ContainerDied","Data":"3933ad44b043460495a01602b638c58b2397fef81da797ff1b5a5e9977cf93f5"}
Sep 08 15:28:48.535108 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.534852    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4" event={"ID":"d8c2924e-19c4-46a2-ba2a-cfbfa5118719","Type":"ContainerDied","Data":"645a809bce4c1841d972edeff2a59d430599af9b6541a8cabda943be1b8ab42d"}
Sep 08 15:28:48.535108 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.534886    2575 scope.go:117] "RemoveContainer" containerID="3933ad44b043460495a01602b638c58b2397fef81da797ff1b5a5e9977cf93f5"
Sep 08 15:28:48.536757 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.536731    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7" event={"ID":"69f24fab-d84d-4958-a99a-bc65fc857a18","Type":"ContainerStarted","Data":"6267fca4bf347f9cb1505c870fa4c57e01770473ea0e26f5f1c69040c6b5b929"}
Sep 08 15:28:48.536878 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.536767    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7" event={"ID":"69f24fab-d84d-4958-a99a-bc65fc857a18","Type":"ContainerStarted","Data":"05e7635d21049b9085f4c3397a6daf1a19d02f39142a0ca592678a09fee240eb"}
Sep 08 15:28:48.543846 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.543829    2575 scope.go:117] "RemoveContainer" containerID="3933ad44b043460495a01602b638c58b2397fef81da797ff1b5a5e9977cf93f5"
Sep 08 15:28:48.544122 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:28:48.544100    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3933ad44b043460495a01602b638c58b2397fef81da797ff1b5a5e9977cf93f5\": container with ID starting with 3933ad44b043460495a01602b638c58b2397fef81da797ff1b5a5e9977cf93f5 not found: ID does not exist" containerID="3933ad44b043460495a01602b638c58b2397fef81da797ff1b5a5e9977cf93f5"
Sep 08 15:28:48.544208 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.544127    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3933ad44b043460495a01602b638c58b2397fef81da797ff1b5a5e9977cf93f5"} err="failed to get container status \"3933ad44b043460495a01602b638c58b2397fef81da797ff1b5a5e9977cf93f5\": rpc error: code = NotFound desc = could not find container \"3933ad44b043460495a01602b638c58b2397fef81da797ff1b5a5e9977cf93f5\": container with ID starting with 3933ad44b043460495a01602b638c58b2397fef81da797ff1b5a5e9977cf93f5 not found: ID does not exist"
Sep 08 15:28:48.556949 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.556902    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"]
Sep 08 15:28:48.558749 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.558731    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-152e35a7-54fdf8d789-ktrt4"]
Sep 08 15:28:48.575978 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.575938    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7" podStartSLOduration=1.5759277969999999 podStartE2EDuration="1.575927797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:28:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:28:48.574033869 +0000 UTC m=+1176.659983911" watchObservedRunningTime="2026-09-08 15:28:48.575927797 +0000 UTC m=+1176.661877838"
Sep 08 15:28:48.953365 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:48.953333    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:49.050421 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:49.050393    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:50.434081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:50.434048    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8c2924e-19c4-46a2-ba2a-cfbfa5118719" path="/var/lib/kubelet/pods/d8c2924e-19c4-46a2-ba2a-cfbfa5118719/volumes"
Sep 08 15:28:50.576749 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:50.576715    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:51.157746 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:51.157711    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:28:53.267269 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.267217    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:53.321684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.321659    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqmhg\" (UniqueName: \"kubernetes.io/projected/6b5a4117-fb0d-4727-9b1e-21c96127d43d-kube-api-access-wqmhg\") pod \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") "
Sep 08 15:28:53.321827 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.321733    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-credential-socket\") pod \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") "
Sep 08 15:28:53.321827 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.321751    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istiod-ca-cert\") pod \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") "
Sep 08 15:28:53.321951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.321921    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-token\") pod \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") "
Sep 08 15:28:53.322015 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.321955    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-workload-socket\") pod \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") "
Sep 08 15:28:53.322015 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.321988    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-podinfo\") pod \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") "
Sep 08 15:28:53.322117 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.322045    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-data\") pod \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") "
Sep 08 15:28:53.322117 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.322057    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6b5a4117-fb0d-4727-9b1e-21c96127d43d" (UID: "6b5a4117-fb0d-4727-9b1e-21c96127d43d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:28:53.322117 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.322067    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-workload-certs\") pod \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") "
Sep 08 15:28:53.322279 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.322115    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6b5a4117-fb0d-4727-9b1e-21c96127d43d" (UID: "6b5a4117-fb0d-4727-9b1e-21c96127d43d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:28:53.322279 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.322134    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-envoy\") pod \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\" (UID: \"6b5a4117-fb0d-4727-9b1e-21c96127d43d\") "
Sep 08 15:28:53.322279 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.322259    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6b5a4117-fb0d-4727-9b1e-21c96127d43d" (UID: "6b5a4117-fb0d-4727-9b1e-21c96127d43d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:28:53.322438 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.322288    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6b5a4117-fb0d-4727-9b1e-21c96127d43d" (UID: "6b5a4117-fb0d-4727-9b1e-21c96127d43d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:28:53.322438 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.322426    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-data" (OuterVolumeSpecName: "istio-data") pod "6b5a4117-fb0d-4727-9b1e-21c96127d43d" (UID: "6b5a4117-fb0d-4727-9b1e-21c96127d43d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:28:53.322618 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.322596    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:53.322699 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.322617    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:53.322699 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.322630    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:53.322699 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.322642    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:53.322699 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.322653    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:53.324653 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.324626    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6b5a4117-fb0d-4727-9b1e-21c96127d43d" (UID: "6b5a4117-fb0d-4727-9b1e-21c96127d43d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:28:53.324749 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.324707    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6b5a4117-fb0d-4727-9b1e-21c96127d43d" (UID: "6b5a4117-fb0d-4727-9b1e-21c96127d43d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:28:53.324830 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.324809    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b5a4117-fb0d-4727-9b1e-21c96127d43d-kube-api-access-wqmhg" (OuterVolumeSpecName: "kube-api-access-wqmhg") pod "6b5a4117-fb0d-4727-9b1e-21c96127d43d" (UID: "6b5a4117-fb0d-4727-9b1e-21c96127d43d"). InnerVolumeSpecName "kube-api-access-wqmhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:28:53.325082 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.325062    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-token" (OuterVolumeSpecName: "istio-token") pod "6b5a4117-fb0d-4727-9b1e-21c96127d43d" (UID: "6b5a4117-fb0d-4727-9b1e-21c96127d43d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:28:53.423919 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.423890    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqmhg\" (UniqueName: \"kubernetes.io/projected/6b5a4117-fb0d-4727-9b1e-21c96127d43d-kube-api-access-wqmhg\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:53.424022 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.423923    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:53.424022 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.423938    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:53.424022 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.423951    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b5a4117-fb0d-4727-9b1e-21c96127d43d-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:28:53.563882 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.563818    2575 generic.go:356] "Generic (PLEG): container finished" podID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerID="336aab2dafa27a6625c09ee72e9fcde5085c6efeef24731673495db74f3aa03f" exitCode=0
Sep 08 15:28:53.563882 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.563864    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"
Sep 08 15:28:53.564026 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.563902    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r" event={"ID":"6b5a4117-fb0d-4727-9b1e-21c96127d43d","Type":"ContainerDied","Data":"336aab2dafa27a6625c09ee72e9fcde5085c6efeef24731673495db74f3aa03f"}
Sep 08 15:28:53.564026 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.563952    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r" event={"ID":"6b5a4117-fb0d-4727-9b1e-21c96127d43d","Type":"ContainerDied","Data":"8bd3086e3de8d37a6161204d5574b909eec4664efe859191ebc42b28c3f32703"}
Sep 08 15:28:53.564026 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.563975    2575 scope.go:117] "RemoveContainer" containerID="336aab2dafa27a6625c09ee72e9fcde5085c6efeef24731673495db74f3aa03f"
Sep 08 15:28:53.572916 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.572898    2575 scope.go:117] "RemoveContainer" containerID="0403effebd631cbc049197837e4003ed2ff25d2dc48da5ecbf239e908485b518"
Sep 08 15:28:53.580351 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.580334    2575 scope.go:117] "RemoveContainer" containerID="336aab2dafa27a6625c09ee72e9fcde5085c6efeef24731673495db74f3aa03f"
Sep 08 15:28:53.580637 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:28:53.580619    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"336aab2dafa27a6625c09ee72e9fcde5085c6efeef24731673495db74f3aa03f\": container with ID starting with 336aab2dafa27a6625c09ee72e9fcde5085c6efeef24731673495db74f3aa03f not found: ID does not exist" containerID="336aab2dafa27a6625c09ee72e9fcde5085c6efeef24731673495db74f3aa03f"
Sep 08 15:28:53.580702 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.580644    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"336aab2dafa27a6625c09ee72e9fcde5085c6efeef24731673495db74f3aa03f"} err="failed to get container status \"336aab2dafa27a6625c09ee72e9fcde5085c6efeef24731673495db74f3aa03f\": rpc error: code = NotFound desc = could not find container \"336aab2dafa27a6625c09ee72e9fcde5085c6efeef24731673495db74f3aa03f\": container with ID starting with 336aab2dafa27a6625c09ee72e9fcde5085c6efeef24731673495db74f3aa03f not found: ID does not exist"
Sep 08 15:28:53.580702 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.580659    2575 scope.go:117] "RemoveContainer" containerID="0403effebd631cbc049197837e4003ed2ff25d2dc48da5ecbf239e908485b518"
Sep 08 15:28:53.580909 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:28:53.580892    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0403effebd631cbc049197837e4003ed2ff25d2dc48da5ecbf239e908485b518\": container with ID starting with 0403effebd631cbc049197837e4003ed2ff25d2dc48da5ecbf239e908485b518 not found: ID does not exist" containerID="0403effebd631cbc049197837e4003ed2ff25d2dc48da5ecbf239e908485b518"
Sep 08 15:28:53.580955 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.580912    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0403effebd631cbc049197837e4003ed2ff25d2dc48da5ecbf239e908485b518"} err="failed to get container status \"0403effebd631cbc049197837e4003ed2ff25d2dc48da5ecbf239e908485b518\": rpc error: code = NotFound desc = could not find container \"0403effebd631cbc049197837e4003ed2ff25d2dc48da5ecbf239e908485b518\": container with ID starting with 0403effebd631cbc049197837e4003ed2ff25d2dc48da5ecbf239e908485b518 not found: ID does not exist"
Sep 08 15:28:53.586210 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.586189    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"]
Sep 08 15:28:53.590839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:53.590819    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-152e35a7-openshift-default-74db6f674d-6b98r"]
Sep 08 15:28:54.431507 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:54.431472    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" path="/var/lib/kubelet/pods/6b5a4117-fb0d-4727-9b1e-21c96127d43d/volumes"
Sep 08 15:28:57.167568 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:57.167538    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:57.249253 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:57.249216    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:57.345121 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:57.345088    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:57.441352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:57.441279    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:57.539594 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:57.539566    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:57.753480 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:57.753415    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:57.753480 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:57.753459    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:57.765642 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:57.765606    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:28:57.894328 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:57.894296    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:28:58.600265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:28:58.600239    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:29:06.938663 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:06.938631    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:29:09.495587 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.495548    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"]
Sep 08 15:29:09.529364 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.529332    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:09.644504 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.644470    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj"]
Sep 08 15:29:09.644924 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.644908    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerName="istio-proxy"
Sep 08 15:29:09.644924 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.644926    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerName="istio-proxy"
Sep 08 15:29:09.645022 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.644935    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8c2924e-19c4-46a2-ba2a-cfbfa5118719" containerName="maas-api"
Sep 08 15:29:09.645022 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.644941    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8c2924e-19c4-46a2-ba2a-cfbfa5118719" containerName="maas-api"
Sep 08 15:29:09.645022 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.644953    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerName="istio-proxy"
Sep 08 15:29:09.645022 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.644958    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerName="istio-proxy"
Sep 08 15:29:09.645022 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.645015    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8c2924e-19c4-46a2-ba2a-cfbfa5118719" containerName="maas-api"
Sep 08 15:29:09.645175 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.645024    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerName="istio-proxy"
Sep 08 15:29:09.645175 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.645033    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b5a4117-fb0d-4727-9b1e-21c96127d43d" containerName="istio-proxy"
Sep 08 15:29:09.647703 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.647689    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj"
Sep 08 15:29:09.651299 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.651279    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8qqvg\""
Sep 08 15:29:09.659059 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.659029    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj"]
Sep 08 15:29:09.763504 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.763428    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15c19105-50a4-4cc0-8305-02dfc30acac6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj\" (UID: \"15c19105-50a4-4cc0-8305-02dfc30acac6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj"
Sep 08 15:29:09.864818 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.864789    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15c19105-50a4-4cc0-8305-02dfc30acac6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj\" (UID: \"15c19105-50a4-4cc0-8305-02dfc30acac6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj"
Sep 08 15:29:09.865359 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.865343    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15c19105-50a4-4cc0-8305-02dfc30acac6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj\" (UID: \"15c19105-50a4-4cc0-8305-02dfc30acac6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj"
Sep 08 15:29:09.897449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.897429    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:09.957096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:09.957071    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj"
Sep 08 15:29:10.230381 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:10.230358    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj"]
Sep 08 15:29:10.232355 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:29:10.232330    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c19105_50a4_4cc0_8305_02dfc30acac6.slice/crio-6e80b298491c19b7ab863cb532cc00200effe0f9a0a5fc4bfa83baa9169e52c6 WatchSource:0}: Error finding container 6e80b298491c19b7ab863cb532cc00200effe0f9a0a5fc4bfa83baa9169e52c6: Status 404 returned error can't find the container with id 6e80b298491c19b7ab863cb532cc00200effe0f9a0a5fc4bfa83baa9169e52c6
Sep 08 15:29:10.648859 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:10.648829    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj" event={"ID":"15c19105-50a4-4cc0-8305-02dfc30acac6","Type":"ContainerStarted","Data":"556c421090e61285fdf83ed2eb87391b3320895c2630f73501f589bba4262552"}
Sep 08 15:29:10.649240 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:10.648889    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj" event={"ID":"15c19105-50a4-4cc0-8305-02dfc30acac6","Type":"ContainerStarted","Data":"6e80b298491c19b7ab863cb532cc00200effe0f9a0a5fc4bfa83baa9169e52c6"}
Sep 08 15:29:10.688107 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:10.688064    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj" podStartSLOduration=1.688050816 podStartE2EDuration="1.688050816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:29:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:29:10.685465895 +0000 UTC m=+1198.771415935" watchObservedRunningTime="2026-09-08 15:29:10.688050816 +0000 UTC m=+1198.774000858"
Sep 08 15:29:11.656563 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:11.656535    2575 generic.go:356] "Generic (PLEG): container finished" podID="15c19105-50a4-4cc0-8305-02dfc30acac6" containerID="556c421090e61285fdf83ed2eb87391b3320895c2630f73501f589bba4262552" exitCode=0
Sep 08 15:29:11.656900 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:11.656624    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj" event={"ID":"15c19105-50a4-4cc0-8305-02dfc30acac6","Type":"ContainerDied","Data":"556c421090e61285fdf83ed2eb87391b3320895c2630f73501f589bba4262552"}
Sep 08 15:29:12.387445 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:12.387415    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:29:12.392281 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:12.392261    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:29:12.786055 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:12.786034    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj"
Sep 08 15:29:12.892123 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:12.892098    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15c19105-50a4-4cc0-8305-02dfc30acac6-maas-api-service-ca\") pod \"15c19105-50a4-4cc0-8305-02dfc30acac6\" (UID: \"15c19105-50a4-4cc0-8305-02dfc30acac6\") "
Sep 08 15:29:12.892417 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:12.892395    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15c19105-50a4-4cc0-8305-02dfc30acac6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "15c19105-50a4-4cc0-8305-02dfc30acac6" (UID: "15c19105-50a4-4cc0-8305-02dfc30acac6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:29:12.993526 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:12.993448    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15c19105-50a4-4cc0-8305-02dfc30acac6-maas-api-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:13.667390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:13.667364    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj"
Sep 08 15:29:13.667551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:13.667413    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj" event={"ID":"15c19105-50a4-4cc0-8305-02dfc30acac6","Type":"ContainerDied","Data":"6e80b298491c19b7ab863cb532cc00200effe0f9a0a5fc4bfa83baa9169e52c6"}
Sep 08 15:29:13.667551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:13.667460    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e80b298491c19b7ab863cb532cc00200effe0f9a0a5fc4bfa83baa9169e52c6"
Sep 08 15:29:20.305552 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.305522    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"]
Sep 08 15:29:20.306042 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.305749    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm" podUID="2688761e-d688-496f-a17b-1b5afece2769" containerName="maas-api" containerID="cri-o://1f51684d72f5582f636559f72ca9543bf16122ced23045987b8a6b4c8fc1f1df" gracePeriod=30
Sep 08 15:29:20.345620 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.345591    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:20.493511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.493460    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:20.563306 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.563255    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:29:20.599794 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.599754    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:20.613812 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.613786    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj"]
Sep 08 15:29:20.618585 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.618563    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-6de9d1-cx5jj"]
Sep 08 15:29:20.659269 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.659250    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2688761e-d688-496f-a17b-1b5afece2769-maas-api-tls\") pod \"2688761e-d688-496f-a17b-1b5afece2769\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") "
Sep 08 15:29:20.659359 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.659296    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2688761e-d688-496f-a17b-1b5afece2769-metrics-certs\") pod \"2688761e-d688-496f-a17b-1b5afece2769\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") "
Sep 08 15:29:20.659359 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.659318    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wrcr\" (UniqueName: \"kubernetes.io/projected/2688761e-d688-496f-a17b-1b5afece2769-kube-api-access-6wrcr\") pod \"2688761e-d688-496f-a17b-1b5afece2769\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") "
Sep 08 15:29:20.659459 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.659442    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2688761e-d688-496f-a17b-1b5afece2769-openshift-service-ca\") pod \"2688761e-d688-496f-a17b-1b5afece2769\" (UID: \"2688761e-d688-496f-a17b-1b5afece2769\") "
Sep 08 15:29:20.659843 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.659812    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2688761e-d688-496f-a17b-1b5afece2769-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2688761e-d688-496f-a17b-1b5afece2769" (UID: "2688761e-d688-496f-a17b-1b5afece2769"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:29:20.661400 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.661345    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2688761e-d688-496f-a17b-1b5afece2769-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2688761e-d688-496f-a17b-1b5afece2769" (UID: "2688761e-d688-496f-a17b-1b5afece2769"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:29:20.661482 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.661381    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2688761e-d688-496f-a17b-1b5afece2769-kube-api-access-6wrcr" (OuterVolumeSpecName: "kube-api-access-6wrcr") pod "2688761e-d688-496f-a17b-1b5afece2769" (UID: "2688761e-d688-496f-a17b-1b5afece2769"). InnerVolumeSpecName "kube-api-access-6wrcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:29:20.661482 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.661441    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2688761e-d688-496f-a17b-1b5afece2769-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2688761e-d688-496f-a17b-1b5afece2769" (UID: "2688761e-d688-496f-a17b-1b5afece2769"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:29:20.686376 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.686355    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:20.707371 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.707342    2575 generic.go:356] "Generic (PLEG): container finished" podID="2688761e-d688-496f-a17b-1b5afece2769" containerID="1f51684d72f5582f636559f72ca9543bf16122ced23045987b8a6b4c8fc1f1df" exitCode=0
Sep 08 15:29:20.707520 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.707393    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"
Sep 08 15:29:20.707520 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.707429    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm" event={"ID":"2688761e-d688-496f-a17b-1b5afece2769","Type":"ContainerDied","Data":"1f51684d72f5582f636559f72ca9543bf16122ced23045987b8a6b4c8fc1f1df"}
Sep 08 15:29:20.707520 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.707480    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm" event={"ID":"2688761e-d688-496f-a17b-1b5afece2769","Type":"ContainerDied","Data":"264d5ee3b180b8f2a867822c5d851c28203c03c8114a82e1cc3d8df26dd3ceac"}
Sep 08 15:29:20.707520 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.707513    2575 scope.go:117] "RemoveContainer" containerID="1f51684d72f5582f636559f72ca9543bf16122ced23045987b8a6b4c8fc1f1df"
Sep 08 15:29:20.716677 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.716661    2575 scope.go:117] "RemoveContainer" containerID="1f51684d72f5582f636559f72ca9543bf16122ced23045987b8a6b4c8fc1f1df"
Sep 08 15:29:20.716952 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:29:20.716935    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f51684d72f5582f636559f72ca9543bf16122ced23045987b8a6b4c8fc1f1df\": container with ID starting with 1f51684d72f5582f636559f72ca9543bf16122ced23045987b8a6b4c8fc1f1df not found: ID does not exist" containerID="1f51684d72f5582f636559f72ca9543bf16122ced23045987b8a6b4c8fc1f1df"
Sep 08 15:29:20.716998 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.716961    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f51684d72f5582f636559f72ca9543bf16122ced23045987b8a6b4c8fc1f1df"} err="failed to get container status \"1f51684d72f5582f636559f72ca9543bf16122ced23045987b8a6b4c8fc1f1df\": rpc error: code = NotFound desc = could not find container \"1f51684d72f5582f636559f72ca9543bf16122ced23045987b8a6b4c8fc1f1df\": container with ID starting with 1f51684d72f5582f636559f72ca9543bf16122ced23045987b8a6b4c8fc1f1df not found: ID does not exist"
Sep 08 15:29:20.734804 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.734780    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"]
Sep 08 15:29:20.736177 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.736157    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-6de9d1-846cf8fd8b-lpbcm"]
Sep 08 15:29:20.760733 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.760714    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2688761e-d688-496f-a17b-1b5afece2769-maas-api-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:20.760733 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.760734    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2688761e-d688-496f-a17b-1b5afece2769-metrics-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:20.760853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.760743    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wrcr\" (UniqueName: \"kubernetes.io/projected/2688761e-d688-496f-a17b-1b5afece2769-kube-api-access-6wrcr\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:20.760853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:20.760752    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2688761e-d688-496f-a17b-1b5afece2769-openshift-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:21.013729 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:21.013702    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:21.266969 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:21.266915    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:29:22.431681 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:22.431648    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15c19105-50a4-4cc0-8305-02dfc30acac6" path="/var/lib/kubelet/pods/15c19105-50a4-4cc0-8305-02dfc30acac6/volumes"
Sep 08 15:29:22.432017 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:22.432004    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2688761e-d688-496f-a17b-1b5afece2769" path="/var/lib/kubelet/pods/2688761e-d688-496f-a17b-1b5afece2769/volumes"
Sep 08 15:29:23.882053 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:23.882020    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:24.514582 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.514549    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7" podUID="69f24fab-d84d-4958-a99a-bc65fc857a18" containerName="main" containerID="cri-o://6267fca4bf347f9cb1505c870fa4c57e01770473ea0e26f5f1c69040c6b5b929" gracePeriod=45
Sep 08 15:29:24.728270 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.728243    2575 generic.go:356] "Generic (PLEG): container finished" podID="69f24fab-d84d-4958-a99a-bc65fc857a18" containerID="6267fca4bf347f9cb1505c870fa4c57e01770473ea0e26f5f1c69040c6b5b929" exitCode=0
Sep 08 15:29:24.728402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.728311    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7" event={"ID":"69f24fab-d84d-4958-a99a-bc65fc857a18","Type":"ContainerDied","Data":"6267fca4bf347f9cb1505c870fa4c57e01770473ea0e26f5f1c69040c6b5b929"}
Sep 08 15:29:24.753217 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.753198    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:29:24.898989 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.898965    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-tmp-dir\") pod \"69f24fab-d84d-4958-a99a-bc65fc857a18\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") "
Sep 08 15:29:24.899377 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.898996    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69f24fab-d84d-4958-a99a-bc65fc857a18-tls-certs\") pod \"69f24fab-d84d-4958-a99a-bc65fc857a18\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") "
Sep 08 15:29:24.899377 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.899028    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-model-cache\") pod \"69f24fab-d84d-4958-a99a-bc65fc857a18\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") "
Sep 08 15:29:24.899377 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.899092    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-home\") pod \"69f24fab-d84d-4958-a99a-bc65fc857a18\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") "
Sep 08 15:29:24.899377 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.899141    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-dshm\") pod \"69f24fab-d84d-4958-a99a-bc65fc857a18\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") "
Sep 08 15:29:24.899377 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.899193    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lk9x\" (UniqueName: \"kubernetes.io/projected/69f24fab-d84d-4958-a99a-bc65fc857a18-kube-api-access-4lk9x\") pod \"69f24fab-d84d-4958-a99a-bc65fc857a18\" (UID: \"69f24fab-d84d-4958-a99a-bc65fc857a18\") "
Sep 08 15:29:24.899377 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.899201    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "69f24fab-d84d-4958-a99a-bc65fc857a18" (UID: "69f24fab-d84d-4958-a99a-bc65fc857a18"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:29:24.899377 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.899338    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-model-cache" (OuterVolumeSpecName: "model-cache") pod "69f24fab-d84d-4958-a99a-bc65fc857a18" (UID: "69f24fab-d84d-4958-a99a-bc65fc857a18"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:29:24.899664 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.899415    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-home" (OuterVolumeSpecName: "home") pod "69f24fab-d84d-4958-a99a-bc65fc857a18" (UID: "69f24fab-d84d-4958-a99a-bc65fc857a18"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:29:24.899664 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.899589    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:24.899664 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.899608    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:24.899664 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.899622    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:24.901160 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.901137    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-dshm" (OuterVolumeSpecName: "dshm") pod "69f24fab-d84d-4958-a99a-bc65fc857a18" (UID: "69f24fab-d84d-4958-a99a-bc65fc857a18"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:29:24.901239 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.901222    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69f24fab-d84d-4958-a99a-bc65fc857a18-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "69f24fab-d84d-4958-a99a-bc65fc857a18" (UID: "69f24fab-d84d-4958-a99a-bc65fc857a18"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:29:24.901302 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:24.901237    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69f24fab-d84d-4958-a99a-bc65fc857a18-kube-api-access-4lk9x" (OuterVolumeSpecName: "kube-api-access-4lk9x") pod "69f24fab-d84d-4958-a99a-bc65fc857a18" (UID: "69f24fab-d84d-4958-a99a-bc65fc857a18"). InnerVolumeSpecName "kube-api-access-4lk9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:29:25.000799 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:25.000774    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69f24fab-d84d-4958-a99a-bc65fc857a18-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:25.000799 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:25.000796    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69f24fab-d84d-4958-a99a-bc65fc857a18-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:25.000928 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:25.000807    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lk9x\" (UniqueName: \"kubernetes.io/projected/69f24fab-d84d-4958-a99a-bc65fc857a18-kube-api-access-4lk9x\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:25.734817 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:25.734792    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"
Sep 08 15:29:25.734978 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:25.734792    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7" event={"ID":"69f24fab-d84d-4958-a99a-bc65fc857a18","Type":"ContainerDied","Data":"05e7635d21049b9085f4c3397a6daf1a19d02f39142a0ca592678a09fee240eb"}
Sep 08 15:29:25.734978 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:25.734910    2575 scope.go:117] "RemoveContainer" containerID="6267fca4bf347f9cb1505c870fa4c57e01770473ea0e26f5f1c69040c6b5b929"
Sep 08 15:29:25.764797 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:25.764775    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"]
Sep 08 15:29:25.766497 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:25.766468    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-6de9d1/explicit-model-6de9d1-kserve-cdf5c78c5-wpjl7"]
Sep 08 15:29:26.432089 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:26.432059    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69f24fab-d84d-4958-a99a-bc65fc857a18" path="/var/lib/kubelet/pods/69f24fab-d84d-4958-a99a-bc65fc857a18/volumes"
Sep 08 15:29:28.216007 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:28.215975    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:28.316709 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:28.316672    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:28.490715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:28.490642    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"]
Sep 08 15:29:28.490902 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:28.490876    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerName="istio-proxy" containerID="cri-o://fec5bb5450245911c6f2eac4d4eba79fcc32fa833334de8e2c605a69321abe45" gracePeriod=30
Sep 08 15:29:28.520175 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:28.520144    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:28.986942 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:28.986908    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:29.409860 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.409829    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"]
Sep 08 15:29:29.410218 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.410205    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69f24fab-d84d-4958-a99a-bc65fc857a18" containerName="main"
Sep 08 15:29:29.410218 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.410218    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="69f24fab-d84d-4958-a99a-bc65fc857a18" containerName="main"
Sep 08 15:29:29.410287 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.410232    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2688761e-d688-496f-a17b-1b5afece2769" containerName="maas-api"
Sep 08 15:29:29.410287 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.410237    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2688761e-d688-496f-a17b-1b5afece2769" containerName="maas-api"
Sep 08 15:29:29.410287 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.410246    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15c19105-50a4-4cc0-8305-02dfc30acac6" containerName="revoke-keys"
Sep 08 15:29:29.410287 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.410252    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="15c19105-50a4-4cc0-8305-02dfc30acac6" containerName="revoke-keys"
Sep 08 15:29:29.410402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.410348    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="69f24fab-d84d-4958-a99a-bc65fc857a18" containerName="main"
Sep 08 15:29:29.410402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.410359    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="15c19105-50a4-4cc0-8305-02dfc30acac6" containerName="revoke-keys"
Sep 08 15:29:29.410402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.410366    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2688761e-d688-496f-a17b-1b5afece2769" containerName="maas-api"
Sep 08 15:29:29.413393 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.413374    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.420255 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.420239    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-342052-openshift-default-dockercfg-twn5h\""
Sep 08 15:29:29.433534 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.433501    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"]
Sep 08 15:29:29.436155 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.436133    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-istio-data\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.436267 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.436177    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-credential-socket\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.436267 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.436218    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c29903f8-e0c2-444f-a194-5326761d54d0-istio-token\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.436387 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.436307    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c29903f8-e0c2-444f-a194-5326761d54d0-istiod-ca-cert\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.436387 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.436369    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-workload-certs\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.436603 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.436576    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c29903f8-e0c2-444f-a194-5326761d54d0-istio-podinfo\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.436710 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.436673    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-workload-socket\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.436763 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.436703    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrmdq\" (UniqueName: \"kubernetes.io/projected/c29903f8-e0c2-444f-a194-5326761d54d0-kube-api-access-lrmdq\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.436807 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.436770    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-istio-envoy\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.474101 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.474082    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:29.537300 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.537271    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c29903f8-e0c2-444f-a194-5326761d54d0-istio-podinfo\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.537446 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.537311    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-workload-socket\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.537446 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.537329    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrmdq\" (UniqueName: \"kubernetes.io/projected/c29903f8-e0c2-444f-a194-5326761d54d0-kube-api-access-lrmdq\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.537446 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.537358    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-istio-envoy\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.537446 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.537391    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-istio-data\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.537446 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.537417    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-credential-socket\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.537446 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.537434    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c29903f8-e0c2-444f-a194-5326761d54d0-istio-token\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.537772 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.537463    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c29903f8-e0c2-444f-a194-5326761d54d0-istiod-ca-cert\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.537772 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.537550    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-workload-certs\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.537870 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.537770    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-workload-socket\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.537870 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.537795    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-credential-socket\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.537939 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.537897    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-istio-data\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.537973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.537935    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-workload-certs\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.538414 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.538391    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c29903f8-e0c2-444f-a194-5326761d54d0-istiod-ca-cert\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.539746 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.539726    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-istio-envoy\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.540065 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.540046    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c29903f8-e0c2-444f-a194-5326761d54d0-istio-podinfo\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.551976 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.551951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c29903f8-e0c2-444f-a194-5326761d54d0-istio-token\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.560049 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.560026    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrmdq\" (UniqueName: \"kubernetes.io/projected/c29903f8-e0c2-444f-a194-5326761d54d0-kube-api-access-lrmdq\") pod \"e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") " pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:29.633241 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.633209    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:29.722900 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:29.722850    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:30.014993 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:30.014930    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"]
Sep 08 15:29:30.018434 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:29:30.018408    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29903f8_e0c2_444f_a194_5326761d54d0.slice/crio-fd00ef7ae7793da54faecaf9a54df7fd89ce98807115a8eefe7fced684713498 WatchSource:0}: Error finding container fd00ef7ae7793da54faecaf9a54df7fd89ce98807115a8eefe7fced684713498: Status 404 returned error can't find the container with id fd00ef7ae7793da54faecaf9a54df7fd89ce98807115a8eefe7fced684713498
Sep 08 15:29:30.020589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:30.020557    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:29:30.020663 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:30.020639    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:29:30.020698 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:30.020668    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:29:30.766910 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:30.766878    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2" event={"ID":"c29903f8-e0c2-444f-a194-5326761d54d0","Type":"ContainerStarted","Data":"4459f8f04ec1b48be372d2055201b9c913cfa85a4c6f0bbd89ed5ede988b325d"}
Sep 08 15:29:30.767337 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:30.766928    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2" event={"ID":"c29903f8-e0c2-444f-a194-5326761d54d0","Type":"ContainerStarted","Data":"fd00ef7ae7793da54faecaf9a54df7fd89ce98807115a8eefe7fced684713498"}
Sep 08 15:29:30.804736 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:30.804687    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2" podStartSLOduration=1.804671036 podStartE2EDuration="1.804671036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:29:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:29:30.803668205 +0000 UTC m=+1218.889618246" watchObservedRunningTime="2026-09-08 15:29:30.804671036 +0000 UTC m=+1218.890621077"
Sep 08 15:29:30.919758 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:30.919697    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:31.020141 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:31.020067    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:31.154909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:31.154872    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:31.265054 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:31.265026    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:31.356591 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:31.356566    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:31.724026 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:31.723954    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:31.728203 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:31.728181    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:31.772380 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:31.772360    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:31.773455 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:31.773436    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:29:31.789596 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:31.789573    2575 patch_prober.go:28] interesting pod/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 15:29:31.789672 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:31.789618    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 15:29:32.851066 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:32.851034    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:29:33.740353 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.740332    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:29:33.771215 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771153    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-data\") pod \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") "
Sep 08 15:29:33.771215 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771195    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdp28\" (UniqueName: \"kubernetes.io/projected/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-kube-api-access-kdp28\") pod \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") "
Sep 08 15:29:33.771383 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771244    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istiod-ca-cert\") pod \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") "
Sep 08 15:29:33.771383 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771280    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-credential-socket\") pod \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") "
Sep 08 15:29:33.771383 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771304    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-workload-socket\") pod \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") "
Sep 08 15:29:33.771383 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771331    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-token\") pod \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") "
Sep 08 15:29:33.771598 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771386    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-data" (OuterVolumeSpecName: "istio-data") pod "07ff4f19-3111-4fe9-934c-40e74d9b5eeb" (UID: "07ff4f19-3111-4fe9-934c-40e74d9b5eeb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:29:33.771598 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771451    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-envoy\") pod \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") "
Sep 08 15:29:33.771598 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771475    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-workload-certs\") pod \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") "
Sep 08 15:29:33.771598 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771543    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-podinfo\") pod \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\" (UID: \"07ff4f19-3111-4fe9-934c-40e74d9b5eeb\") "
Sep 08 15:29:33.771598 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771578    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07ff4f19-3111-4fe9-934c-40e74d9b5eeb" (UID: "07ff4f19-3111-4fe9-934c-40e74d9b5eeb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:29:33.771830 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771612    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07ff4f19-3111-4fe9-934c-40e74d9b5eeb" (UID: "07ff4f19-3111-4fe9-934c-40e74d9b5eeb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:29:33.771830 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771637    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07ff4f19-3111-4fe9-934c-40e74d9b5eeb" (UID: "07ff4f19-3111-4fe9-934c-40e74d9b5eeb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:29:33.771936 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771825    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07ff4f19-3111-4fe9-934c-40e74d9b5eeb" (UID: "07ff4f19-3111-4fe9-934c-40e74d9b5eeb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:29:33.771936 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771902    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:33.771936 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771920    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:33.771936 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771932    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:33.772109 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771944    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:33.772109 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.771958    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:33.773827 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.773804    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07ff4f19-3111-4fe9-934c-40e74d9b5eeb" (UID: "07ff4f19-3111-4fe9-934c-40e74d9b5eeb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:29:33.773921 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.773852    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-kube-api-access-kdp28" (OuterVolumeSpecName: "kube-api-access-kdp28") pod "07ff4f19-3111-4fe9-934c-40e74d9b5eeb" (UID: "07ff4f19-3111-4fe9-934c-40e74d9b5eeb"). InnerVolumeSpecName "kube-api-access-kdp28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:29:33.773982 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.773958    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-token" (OuterVolumeSpecName: "istio-token") pod "07ff4f19-3111-4fe9-934c-40e74d9b5eeb" (UID: "07ff4f19-3111-4fe9-934c-40e74d9b5eeb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:29:33.773982 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.773965    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07ff4f19-3111-4fe9-934c-40e74d9b5eeb" (UID: "07ff4f19-3111-4fe9-934c-40e74d9b5eeb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:29:33.785419 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.785398    2575 generic.go:356] "Generic (PLEG): container finished" podID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerID="fec5bb5450245911c6f2eac4d4eba79fcc32fa833334de8e2c605a69321abe45" exitCode=0
Sep 08 15:29:33.785522 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.785449    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"
Sep 08 15:29:33.785522 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.785475    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" event={"ID":"07ff4f19-3111-4fe9-934c-40e74d9b5eeb","Type":"ContainerDied","Data":"fec5bb5450245911c6f2eac4d4eba79fcc32fa833334de8e2c605a69321abe45"}
Sep 08 15:29:33.785641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.785541    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v" event={"ID":"07ff4f19-3111-4fe9-934c-40e74d9b5eeb","Type":"ContainerDied","Data":"0b0e6bc3ad4e908c11ae0d97578e65da546a7fd722c9b427102e2514592106b3"}
Sep 08 15:29:33.785641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.785563    2575 scope.go:117] "RemoveContainer" containerID="fec5bb5450245911c6f2eac4d4eba79fcc32fa833334de8e2c605a69321abe45"
Sep 08 15:29:33.800251 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.800231    2575 scope.go:117] "RemoveContainer" containerID="a77bfc524ca3b30c923ab1efc9e26eb66e4762021808bbf84067b9cf8d876366"
Sep 08 15:29:33.808183 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.808167    2575 scope.go:117] "RemoveContainer" containerID="fec5bb5450245911c6f2eac4d4eba79fcc32fa833334de8e2c605a69321abe45"
Sep 08 15:29:33.808439 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:29:33.808420    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fec5bb5450245911c6f2eac4d4eba79fcc32fa833334de8e2c605a69321abe45\": container with ID starting with fec5bb5450245911c6f2eac4d4eba79fcc32fa833334de8e2c605a69321abe45 not found: ID does not exist" containerID="fec5bb5450245911c6f2eac4d4eba79fcc32fa833334de8e2c605a69321abe45"
Sep 08 15:29:33.808513 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.808446    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fec5bb5450245911c6f2eac4d4eba79fcc32fa833334de8e2c605a69321abe45"} err="failed to get container status \"fec5bb5450245911c6f2eac4d4eba79fcc32fa833334de8e2c605a69321abe45\": rpc error: code = NotFound desc = could not find container \"fec5bb5450245911c6f2eac4d4eba79fcc32fa833334de8e2c605a69321abe45\": container with ID starting with fec5bb5450245911c6f2eac4d4eba79fcc32fa833334de8e2c605a69321abe45 not found: ID does not exist"
Sep 08 15:29:33.808513 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.808463    2575 scope.go:117] "RemoveContainer" containerID="a77bfc524ca3b30c923ab1efc9e26eb66e4762021808bbf84067b9cf8d876366"
Sep 08 15:29:33.808715 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:29:33.808699    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a77bfc524ca3b30c923ab1efc9e26eb66e4762021808bbf84067b9cf8d876366\": container with ID starting with a77bfc524ca3b30c923ab1efc9e26eb66e4762021808bbf84067b9cf8d876366 not found: ID does not exist" containerID="a77bfc524ca3b30c923ab1efc9e26eb66e4762021808bbf84067b9cf8d876366"
Sep 08 15:29:33.808756 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.808720    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a77bfc524ca3b30c923ab1efc9e26eb66e4762021808bbf84067b9cf8d876366"} err="failed to get container status \"a77bfc524ca3b30c923ab1efc9e26eb66e4762021808bbf84067b9cf8d876366\": rpc error: code = NotFound desc = could not find container \"a77bfc524ca3b30c923ab1efc9e26eb66e4762021808bbf84067b9cf8d876366\": container with ID starting with a77bfc524ca3b30c923ab1efc9e26eb66e4762021808bbf84067b9cf8d876366 not found: ID does not exist"
Sep 08 15:29:33.817408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.817387    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"]
Sep 08 15:29:33.826449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.826428    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-6de9d1-openshift-default-7bd76c7c96-qsf2v"]
Sep 08 15:29:33.873025 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.873006    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdp28\" (UniqueName: \"kubernetes.io/projected/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-kube-api-access-kdp28\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:33.873025 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.873025    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:33.873327 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.873034    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:33.873327 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:33.873043    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ff4f19-3111-4fe9-934c-40e74d9b5eeb-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:29:34.437585 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:34.437547    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" path="/var/lib/kubelet/pods/07ff4f19-3111-4fe9-934c-40e74d9b5eeb/volumes"
Sep 08 15:29:47.095623 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:47.095586    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:51.092922 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.092893    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"]
Sep 08 15:29:51.093259 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.093248    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerName="istio-proxy"
Sep 08 15:29:51.093259 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.093259    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerName="istio-proxy"
Sep 08 15:29:51.093328 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.093279    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerName="istio-proxy"
Sep 08 15:29:51.093328 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.093285    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerName="istio-proxy"
Sep 08 15:29:51.093393 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.093347    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerName="istio-proxy"
Sep 08 15:29:51.093393 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.093357    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="07ff4f19-3111-4fe9-934c-40e74d9b5eeb" containerName="istio-proxy"
Sep 08 15:29:51.096161 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.096145    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.099565 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.099543    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-342052\"/\"noten-model-342052-kserve-self-signed-certs\""
Sep 08 15:29:51.100600 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.100584    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-342052\"/\"openshift-service-ca.crt\""
Sep 08 15:29:51.101915 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.101890    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-342052\"/\"default-dockercfg-hzp5c\""
Sep 08 15:29:51.102515 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.102230    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-342052\"/\"kube-root-ca.crt\""
Sep 08 15:29:51.116097 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.116071    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"]
Sep 08 15:29:51.132788 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.132759    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:51.205229 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.205205    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-model-cache\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.205321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.205245    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tmp-dir\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.205321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.205290    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-dshm\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.205406 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.205334    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-home\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.205406 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.205378    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crk8m\" (UniqueName: \"kubernetes.io/projected/6c82580c-f272-40ef-8b12-ca4d242d9e9f-kube-api-access-crk8m\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.205406 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.205398    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.306261 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.306232    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crk8m\" (UniqueName: \"kubernetes.io/projected/6c82580c-f272-40ef-8b12-ca4d242d9e9f-kube-api-access-crk8m\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.306385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.306273    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.306385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.306342    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-model-cache\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.306529 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.306382    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tmp-dir\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.306529 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.306427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-dshm\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.306529 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.306462    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-home\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.306802 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.306784    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tmp-dir\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.306878 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.306861    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-home\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.306935 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.306883    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-model-cache\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.308638 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.308618    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-dshm\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.308809 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.308791    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.320459 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.320440    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crk8m\" (UniqueName: \"kubernetes.io/projected/6c82580c-f272-40ef-8b12-ca4d242d9e9f-kube-api-access-crk8m\") pod \"noten-model-342052-kserve-567c4f4b7f-4z4hn\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") " pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.405291 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.405270    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:29:51.678480 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.678454    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"]
Sep 08 15:29:51.680849 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:29:51.680819    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c82580c_f272_40ef_8b12_ca4d242d9e9f.slice/crio-9a3db94de788c0af389b3fd221529c10ac85500c4f3e2ba7494c0ed062f9045b WatchSource:0}: Error finding container 9a3db94de788c0af389b3fd221529c10ac85500c4f3e2ba7494c0ed062f9045b: Status 404 returned error can't find the container with id 9a3db94de788c0af389b3fd221529c10ac85500c4f3e2ba7494c0ed062f9045b
Sep 08 15:29:51.882979 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.882945    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn" event={"ID":"6c82580c-f272-40ef-8b12-ca4d242d9e9f","Type":"ContainerStarted","Data":"51edd4ebcc15ee666eedc80c952d4b9b7aded5e1a55afe1fad08091b23f05bae"}
Sep 08 15:29:51.883108 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.882987    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn" event={"ID":"6c82580c-f272-40ef-8b12-ca4d242d9e9f","Type":"ContainerStarted","Data":"9a3db94de788c0af389b3fd221529c10ac85500c4f3e2ba7494c0ed062f9045b"}
Sep 08 15:29:51.908062 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:51.908012    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn" podStartSLOduration=0.907997577 podStartE2EDuration="907.997577ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:29:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:29:51.906344184 +0000 UTC m=+1239.992294226" watchObservedRunningTime="2026-09-08 15:29:51.907997577 +0000 UTC m=+1239.993947664"
Sep 08 15:29:53.602511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:53.602465    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:53.700167 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:53.700141    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:53.901230 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:53.901156    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:29:54.001835 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:29:54.001809    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:00.208147 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.208114    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc"]
Sep 08 15:30:00.211633 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.211616    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc"
Sep 08 15:30:00.218957 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.218939    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8qqvg\""
Sep 08 15:30:00.221026 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.221009    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc"
Sep 08 15:30:00.229247 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.229204    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7"]
Sep 08 15:30:00.242045 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.242027    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7"
Sep 08 15:30:00.243433 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.243381    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc"]
Sep 08 15:30:00.252120 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.252099    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7"]
Sep 08 15:30:00.260258 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.260240    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7"
Sep 08 15:30:00.552348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.552322    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc"]
Sep 08 15:30:00.553833 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:30:00.553805    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba24b3c_5002_458d_b2f8_827b204368a1.slice/crio-4e2daf995d6c19e4b3b3402ea95537d9c7b6bbb6fbf62ab95a818c045ed2b757 WatchSource:0}: Error finding container 4e2daf995d6c19e4b3b3402ea95537d9c7b6bbb6fbf62ab95a818c045ed2b757: Status 404 returned error can't find the container with id 4e2daf995d6c19e4b3b3402ea95537d9c7b6bbb6fbf62ab95a818c045ed2b757
Sep 08 15:30:00.585431 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.585412    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7"]
Sep 08 15:30:00.587563 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:30:00.587536    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2304fb8_8796_4a20_ab9c_b03fb4f34ed3.slice/crio-ce3db6624c9d13edb264e06a03d29e0271d2ab8ab9ec50f489c2d9ca2da4e928 WatchSource:0}: Error finding container ce3db6624c9d13edb264e06a03d29e0271d2ab8ab9ec50f489c2d9ca2da4e928: Status 404 returned error can't find the container with id ce3db6624c9d13edb264e06a03d29e0271d2ab8ab9ec50f489c2d9ca2da4e928
Sep 08 15:30:00.928502 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.928460    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc" event={"ID":"dba24b3c-5002-458d-b2f8-827b204368a1","Type":"ContainerStarted","Data":"9cb9e69e65ea3a41ed29d025090f676b71587c3cd259d600720f09f17344ad31"}
Sep 08 15:30:00.928612 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.928536    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc" event={"ID":"dba24b3c-5002-458d-b2f8-827b204368a1","Type":"ContainerStarted","Data":"4e2daf995d6c19e4b3b3402ea95537d9c7b6bbb6fbf62ab95a818c045ed2b757"}
Sep 08 15:30:00.929958 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.929938    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7" event={"ID":"d2304fb8-8796-4a20-ab9c-b03fb4f34ed3","Type":"ContainerStarted","Data":"e97f8fbc4f841eedd4ca7572a141e6a81ef1dd983645e7666032b8fa2fc83b8c"}
Sep 08 15:30:00.930041 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.929975    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7" event={"ID":"d2304fb8-8796-4a20-ab9c-b03fb4f34ed3","Type":"ContainerStarted","Data":"ce3db6624c9d13edb264e06a03d29e0271d2ab8ab9ec50f489c2d9ca2da4e928"}
Sep 08 15:30:00.986399 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:00.986358    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc" podStartSLOduration=0.986344347 podStartE2EDuration="986.344347ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:00.985941545 +0000 UTC m=+1249.071891585" watchObservedRunningTime="2026-09-08 15:30:00.986344347 +0000 UTC m=+1249.072294387"
Sep 08 15:30:01.022821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:01.022778    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7" podStartSLOduration=1.022764602 podStartE2EDuration="1.022764602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:01.02141482 +0000 UTC m=+1249.107364861" watchObservedRunningTime="2026-09-08 15:30:01.022764602 +0000 UTC m=+1249.108714655"
Sep 08 15:30:01.406085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:01.406060    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:30:01.406376 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:01.406105    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:30:01.418375 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:01.418347    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:30:01.936048 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:01.936022    2575 generic.go:356] "Generic (PLEG): container finished" podID="dba24b3c-5002-458d-b2f8-827b204368a1" containerID="9cb9e69e65ea3a41ed29d025090f676b71587c3cd259d600720f09f17344ad31" exitCode=0
Sep 08 15:30:01.936284 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:01.936111    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc" event={"ID":"dba24b3c-5002-458d-b2f8-827b204368a1","Type":"ContainerDied","Data":"9cb9e69e65ea3a41ed29d025090f676b71587c3cd259d600720f09f17344ad31"}
Sep 08 15:30:01.937611 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:01.937593    2575 generic.go:356] "Generic (PLEG): container finished" podID="d2304fb8-8796-4a20-ab9c-b03fb4f34ed3" containerID="e97f8fbc4f841eedd4ca7572a141e6a81ef1dd983645e7666032b8fa2fc83b8c" exitCode=0
Sep 08 15:30:01.937683 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:01.937623    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7" event={"ID":"d2304fb8-8796-4a20-ab9c-b03fb4f34ed3","Type":"ContainerDied","Data":"e97f8fbc4f841eedd4ca7572a141e6a81ef1dd983645e7666032b8fa2fc83b8c"}
Sep 08 15:30:01.948653 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:01.948638    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:30:01.991038 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:01.991013    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-342052/noten-model-342052-kserve-self-signed-certs: secret "noten-model-342052-kserve-self-signed-certs" not found
Sep 08 15:30:01.991153 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:01.991089    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs podName:6c82580c-f272-40ef-8b12-ca4d242d9e9f nodeName:}" failed. No retries permitted until 2026-09-08 15:30:02.491069168 +0000 UTC m=+1250.577019192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs") pod "noten-model-342052-kserve-567c4f4b7f-4z4hn" (UID: "6c82580c-f272-40ef-8b12-ca4d242d9e9f") : secret "noten-model-342052-kserve-self-signed-certs" not found
Sep 08 15:30:02.079749 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:02.079719    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"]
Sep 08 15:30:02.237790 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:02.237722    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"]
Sep 08 15:30:02.237922 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:02.237900    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2" podUID="c29903f8-e0c2-444f-a194-5326761d54d0" containerName="istio-proxy" containerID="cri-o://4459f8f04ec1b48be372d2055201b9c913cfa85a4c6f0bbd89ed5ede988b325d" gracePeriod=30
Sep 08 15:30:02.267675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:02.267647    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:02.495044 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:02.494958    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-342052/noten-model-342052-kserve-self-signed-certs: secret "noten-model-342052-kserve-self-signed-certs" not found
Sep 08 15:30:02.495044 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:02.495028    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs podName:6c82580c-f272-40ef-8b12-ca4d242d9e9f nodeName:}" failed. No retries permitted until 2026-09-08 15:30:03.495011759 +0000 UTC m=+1251.580961779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs") pod "noten-model-342052-kserve-567c4f4b7f-4z4hn" (UID: "6c82580c-f272-40ef-8b12-ca4d242d9e9f") : secret "noten-model-342052-kserve-self-signed-certs" not found
Sep 08 15:30:02.812253 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:02.812177    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:02.937553 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:02.937523    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:03.074234 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:03.074179    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7"
Sep 08 15:30:03.100360 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:03.100337    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc"
Sep 08 15:30:03.504528 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:03.504483    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-342052/noten-model-342052-kserve-self-signed-certs: secret "noten-model-342052-kserve-self-signed-certs" not found
Sep 08 15:30:03.504844 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:03.504561    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs podName:6c82580c-f272-40ef-8b12-ca4d242d9e9f nodeName:}" failed. No retries permitted until 2026-09-08 15:30:05.504545576 +0000 UTC m=+1253.590495595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs") pod "noten-model-342052-kserve-567c4f4b7f-4z4hn" (UID: "6c82580c-f272-40ef-8b12-ca4d242d9e9f") : secret "noten-model-342052-kserve-self-signed-certs" not found
Sep 08 15:30:03.947586 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:03.947555    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc"
Sep 08 15:30:03.947748 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:03.947560    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc" event={"ID":"dba24b3c-5002-458d-b2f8-827b204368a1","Type":"ContainerDied","Data":"4e2daf995d6c19e4b3b3402ea95537d9c7b6bbb6fbf62ab95a818c045ed2b757"}
Sep 08 15:30:03.947748 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:03.947685    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e2daf995d6c19e4b3b3402ea95537d9c7b6bbb6fbf62ab95a818c045ed2b757"
Sep 08 15:30:03.948811 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:03.948788    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7" event={"ID":"d2304fb8-8796-4a20-ab9c-b03fb4f34ed3","Type":"ContainerDied","Data":"ce3db6624c9d13edb264e06a03d29e0271d2ab8ab9ec50f489c2d9ca2da4e928"}
Sep 08 15:30:03.948911 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:03.948834    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce3db6624c9d13edb264e06a03d29e0271d2ab8ab9ec50f489c2d9ca2da4e928"
Sep 08 15:30:03.948911 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:03.948845    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7"
Sep 08 15:30:04.940278 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:04.940206    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:05.193818 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:05.193749    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"]
Sep 08 15:30:05.521598 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:05.521511    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-342052/noten-model-342052-kserve-self-signed-certs: secret "noten-model-342052-kserve-self-signed-certs" not found
Sep 08 15:30:05.521598 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:05.521581    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs podName:6c82580c-f272-40ef-8b12-ca4d242d9e9f nodeName:}" failed. No retries permitted until 2026-09-08 15:30:09.521566149 +0000 UTC m=+1257.607516169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs") pod "noten-model-342052-kserve-567c4f4b7f-4z4hn" (UID: "6c82580c-f272-40ef-8b12-ca4d242d9e9f") : secret "noten-model-342052-kserve-self-signed-certs" not found
Sep 08 15:30:05.610901 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:05.610873    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:06.403113 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.403084    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"]
Sep 08 15:30:06.599997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.599968    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp"]
Sep 08 15:30:06.600335 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.600323    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba24b3c-5002-458d-b2f8-827b204368a1" containerName="cleanup"
Sep 08 15:30:06.600381 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.600336    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba24b3c-5002-458d-b2f8-827b204368a1" containerName="cleanup"
Sep 08 15:30:06.600381 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.600346    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2304fb8-8796-4a20-ab9c-b03fb4f34ed3" containerName="cleanup"
Sep 08 15:30:06.600381 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.600353    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2304fb8-8796-4a20-ab9c-b03fb4f34ed3" containerName="cleanup"
Sep 08 15:30:06.600544 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.600413    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dba24b3c-5002-458d-b2f8-827b204368a1" containerName="cleanup"
Sep 08 15:30:06.600544 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.600421    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2304fb8-8796-4a20-ab9c-b03fb4f34ed3" containerName="cleanup"
Sep 08 15:30:06.604711 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.604677    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp"
Sep 08 15:30:06.620972 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.620948    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8qqvg\""
Sep 08 15:30:06.620972 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.620967    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:30:06.621690 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.621671    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:06.697894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.697834    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp"]
Sep 08 15:30:06.736910 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.736880    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee2b102-0798-477d-80af-8ed785152dc5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp\" (UID: \"dee2b102-0798-477d-80af-8ed785152dc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp"
Sep 08 15:30:06.837534 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.837485    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee2b102-0798-477d-80af-8ed785152dc5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp\" (UID: \"dee2b102-0798-477d-80af-8ed785152dc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp"
Sep 08 15:30:06.838106 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.838087    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee2b102-0798-477d-80af-8ed785152dc5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp\" (UID: \"dee2b102-0798-477d-80af-8ed785152dc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp"
Sep 08 15:30:06.873873 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.873844    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:06.914299 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:06.914280    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp"
Sep 08 15:30:07.229695 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.229664    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp"]
Sep 08 15:30:07.229900 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:30:07.229878    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddee2b102_0798_477d_80af_8ed785152dc5.slice/crio-1ef7b8858a66c8ac9ac908fc6f83ad18e3856a8ef32d2364ac5607ef65d1c1c4 WatchSource:0}: Error finding container 1ef7b8858a66c8ac9ac908fc6f83ad18e3856a8ef32d2364ac5607ef65d1c1c4: Status 404 returned error can't find the container with id 1ef7b8858a66c8ac9ac908fc6f83ad18e3856a8ef32d2364ac5607ef65d1c1c4
Sep 08 15:30:07.455473 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.455448    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:30:07.544798 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.544767    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c29903f8-e0c2-444f-a194-5326761d54d0-istio-token\") pod \"c29903f8-e0c2-444f-a194-5326761d54d0\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") "
Sep 08 15:30:07.544907 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.544883    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-workload-socket\") pod \"c29903f8-e0c2-444f-a194-5326761d54d0\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") "
Sep 08 15:30:07.544963 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.544932    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-istio-data\") pod \"c29903f8-e0c2-444f-a194-5326761d54d0\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") "
Sep 08 15:30:07.545019 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.544976    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c29903f8-e0c2-444f-a194-5326761d54d0-istio-podinfo\") pod \"c29903f8-e0c2-444f-a194-5326761d54d0\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") "
Sep 08 15:30:07.545019 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.545015    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c29903f8-e0c2-444f-a194-5326761d54d0-istiod-ca-cert\") pod \"c29903f8-e0c2-444f-a194-5326761d54d0\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") "
Sep 08 15:30:07.545121 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.545045    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-workload-certs\") pod \"c29903f8-e0c2-444f-a194-5326761d54d0\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") "
Sep 08 15:30:07.545121 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.545084    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrmdq\" (UniqueName: \"kubernetes.io/projected/c29903f8-e0c2-444f-a194-5326761d54d0-kube-api-access-lrmdq\") pod \"c29903f8-e0c2-444f-a194-5326761d54d0\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") "
Sep 08 15:30:07.545121 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.545111    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-istio-envoy\") pod \"c29903f8-e0c2-444f-a194-5326761d54d0\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") "
Sep 08 15:30:07.545274 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.545150    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-credential-socket\") pod \"c29903f8-e0c2-444f-a194-5326761d54d0\" (UID: \"c29903f8-e0c2-444f-a194-5326761d54d0\") "
Sep 08 15:30:07.545274 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.545180    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c29903f8-e0c2-444f-a194-5326761d54d0" (UID: "c29903f8-e0c2-444f-a194-5326761d54d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:07.545274 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.545256    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-istio-data" (OuterVolumeSpecName: "istio-data") pod "c29903f8-e0c2-444f-a194-5326761d54d0" (UID: "c29903f8-e0c2-444f-a194-5326761d54d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:07.545577 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.545418    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c29903f8-e0c2-444f-a194-5326761d54d0" (UID: "c29903f8-e0c2-444f-a194-5326761d54d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:07.545577 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.545513    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c29903f8-e0c2-444f-a194-5326761d54d0" (UID: "c29903f8-e0c2-444f-a194-5326761d54d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:07.545577 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.545534    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29903f8-e0c2-444f-a194-5326761d54d0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c29903f8-e0c2-444f-a194-5326761d54d0" (UID: "c29903f8-e0c2-444f-a194-5326761d54d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:30:07.545758 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.545597    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:07.545758 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.545616    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:07.545758 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.545630    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:07.547081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.547061    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c29903f8-e0c2-444f-a194-5326761d54d0-istio-token" (OuterVolumeSpecName: "istio-token") pod "c29903f8-e0c2-444f-a194-5326761d54d0" (UID: "c29903f8-e0c2-444f-a194-5326761d54d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:30:07.547259 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.547240    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c29903f8-e0c2-444f-a194-5326761d54d0-kube-api-access-lrmdq" (OuterVolumeSpecName: "kube-api-access-lrmdq") pod "c29903f8-e0c2-444f-a194-5326761d54d0" (UID: "c29903f8-e0c2-444f-a194-5326761d54d0"). InnerVolumeSpecName "kube-api-access-lrmdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:30:07.547527 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.547505    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c29903f8-e0c2-444f-a194-5326761d54d0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c29903f8-e0c2-444f-a194-5326761d54d0" (UID: "c29903f8-e0c2-444f-a194-5326761d54d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:30:07.547608 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.547511    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c29903f8-e0c2-444f-a194-5326761d54d0" (UID: "c29903f8-e0c2-444f-a194-5326761d54d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:07.646659 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.646630    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c29903f8-e0c2-444f-a194-5326761d54d0-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:07.646659 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.646655    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c29903f8-e0c2-444f-a194-5326761d54d0-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:07.646797 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.646668    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrmdq\" (UniqueName: \"kubernetes.io/projected/c29903f8-e0c2-444f-a194-5326761d54d0-kube-api-access-lrmdq\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:07.646797 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.646680    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:07.646797 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.646691    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c29903f8-e0c2-444f-a194-5326761d54d0-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:07.646797 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.646703    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c29903f8-e0c2-444f-a194-5326761d54d0-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:07.875031 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.874996    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:30:07.972164 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.972132    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp" event={"ID":"dee2b102-0798-477d-80af-8ed785152dc5","Type":"ContainerStarted","Data":"ebea5f35db3c5dd241d7d8d16f07914dfd333d7c1139b7cb696867483f69e88f"}
Sep 08 15:30:07.972302 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.972185    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp" event={"ID":"dee2b102-0798-477d-80af-8ed785152dc5","Type":"ContainerStarted","Data":"1ef7b8858a66c8ac9ac908fc6f83ad18e3856a8ef32d2364ac5607ef65d1c1c4"}
Sep 08 15:30:07.973451 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.973429    2575 generic.go:356] "Generic (PLEG): container finished" podID="c29903f8-e0c2-444f-a194-5326761d54d0" containerID="4459f8f04ec1b48be372d2055201b9c913cfa85a4c6f0bbd89ed5ede988b325d" exitCode=0
Sep 08 15:30:07.973562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.973482    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"
Sep 08 15:30:07.973562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.973513    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2" event={"ID":"c29903f8-e0c2-444f-a194-5326761d54d0","Type":"ContainerDied","Data":"4459f8f04ec1b48be372d2055201b9c913cfa85a4c6f0bbd89ed5ede988b325d"}
Sep 08 15:30:07.973663 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.973561    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2" event={"ID":"c29903f8-e0c2-444f-a194-5326761d54d0","Type":"ContainerDied","Data":"fd00ef7ae7793da54faecaf9a54df7fd89ce98807115a8eefe7fced684713498"}
Sep 08 15:30:07.973663 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.973588    2575 scope.go:117] "RemoveContainer" containerID="4459f8f04ec1b48be372d2055201b9c913cfa85a4c6f0bbd89ed5ede988b325d"
Sep 08 15:30:07.982814 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.982796    2575 scope.go:117] "RemoveContainer" containerID="4459f8f04ec1b48be372d2055201b9c913cfa85a4c6f0bbd89ed5ede988b325d"
Sep 08 15:30:07.983082 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:07.983064    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4459f8f04ec1b48be372d2055201b9c913cfa85a4c6f0bbd89ed5ede988b325d\": container with ID starting with 4459f8f04ec1b48be372d2055201b9c913cfa85a4c6f0bbd89ed5ede988b325d not found: ID does not exist" containerID="4459f8f04ec1b48be372d2055201b9c913cfa85a4c6f0bbd89ed5ede988b325d"
Sep 08 15:30:07.983147 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:07.983089    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4459f8f04ec1b48be372d2055201b9c913cfa85a4c6f0bbd89ed5ede988b325d"} err="failed to get container status \"4459f8f04ec1b48be372d2055201b9c913cfa85a4c6f0bbd89ed5ede988b325d\": rpc error: code = NotFound desc = could not find container \"4459f8f04ec1b48be372d2055201b9c913cfa85a4c6f0bbd89ed5ede988b325d\": container with ID starting with 4459f8f04ec1b48be372d2055201b9c913cfa85a4c6f0bbd89ed5ede988b325d not found: ID does not exist"
Sep 08 15:30:08.013293 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:08.013266    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:08.097927 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:08.097881    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp" podStartSLOduration=2.097867691 podStartE2EDuration="2.097867691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:08.09383569 +0000 UTC m=+1256.179785736" watchObservedRunningTime="2026-09-08 15:30:08.097867691 +0000 UTC m=+1256.183817730"
Sep 08 15:30:08.145231 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:08.145175    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"]
Sep 08 15:30:08.160477 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:08.160449    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-342052-openshift-default-64f8dcdb7b-5fns2"]
Sep 08 15:30:08.432193 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:08.432140    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c29903f8-e0c2-444f-a194-5326761d54d0" path="/var/lib/kubelet/pods/c29903f8-e0c2-444f-a194-5326761d54d0/volumes"
Sep 08 15:30:08.980020 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:08.979989    2575 generic.go:356] "Generic (PLEG): container finished" podID="dee2b102-0798-477d-80af-8ed785152dc5" containerID="ebea5f35db3c5dd241d7d8d16f07914dfd333d7c1139b7cb696867483f69e88f" exitCode=0
Sep 08 15:30:08.980361 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:08.980071    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp" event={"ID":"dee2b102-0798-477d-80af-8ed785152dc5","Type":"ContainerDied","Data":"ebea5f35db3c5dd241d7d8d16f07914dfd333d7c1139b7cb696867483f69e88f"}
Sep 08 15:30:09.563592 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:09.563565    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-342052/noten-model-342052-kserve-self-signed-certs: secret "noten-model-342052-kserve-self-signed-certs" not found
Sep 08 15:30:09.563747 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:09.563624    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs podName:6c82580c-f272-40ef-8b12-ca4d242d9e9f nodeName:}" failed. No retries permitted until 2026-09-08 15:30:17.563610093 +0000 UTC m=+1265.649560113 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs") pod "noten-model-342052-kserve-567c4f4b7f-4z4hn" (UID: "6c82580c-f272-40ef-8b12-ca4d242d9e9f") : secret "noten-model-342052-kserve-self-signed-certs" not found
Sep 08 15:30:10.110831 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:10.110809    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp"
Sep 08 15:30:10.170231 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:10.170206    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee2b102-0798-477d-80af-8ed785152dc5-maas-api-service-ca\") pod \"dee2b102-0798-477d-80af-8ed785152dc5\" (UID: \"dee2b102-0798-477d-80af-8ed785152dc5\") "
Sep 08 15:30:10.170593 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:10.170567    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dee2b102-0798-477d-80af-8ed785152dc5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dee2b102-0798-477d-80af-8ed785152dc5" (UID: "dee2b102-0798-477d-80af-8ed785152dc5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:30:10.271677 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:10.271623    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dee2b102-0798-477d-80af-8ed785152dc5-maas-api-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:10.874962 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:10.874929    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:10.998142 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:10.998116    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp"
Sep 08 15:30:10.998306 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:10.998137    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp" event={"ID":"dee2b102-0798-477d-80af-8ed785152dc5","Type":"ContainerDied","Data":"1ef7b8858a66c8ac9ac908fc6f83ad18e3856a8ef32d2364ac5607ef65d1c1c4"}
Sep 08 15:30:10.998306 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:10.998198    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ef7b8858a66c8ac9ac908fc6f83ad18e3856a8ef32d2364ac5607ef65d1c1c4"
Sep 08 15:30:17.642599 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:17.642572    2575 secret.go:189] Couldn't get secret ai-tenant-e2e-noten-342052/noten-model-342052-kserve-self-signed-certs: secret "noten-model-342052-kserve-self-signed-certs" not found
Sep 08 15:30:17.642957 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:17.642630    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs podName:6c82580c-f272-40ef-8b12-ca4d242d9e9f nodeName:}" failed. No retries permitted until 2026-09-08 15:30:33.642617882 +0000 UTC m=+1281.728567901 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs") pod "noten-model-342052-kserve-567c4f4b7f-4z4hn" (UID: "6c82580c-f272-40ef-8b12-ca4d242d9e9f") : secret "noten-model-342052-kserve-self-signed-certs" not found
Sep 08 15:30:17.990290 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:17.990216    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7"]
Sep 08 15:30:18.000042 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:18.000015    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-d8e721-29814690-5nbr7"]
Sep 08 15:30:18.026322 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:18.026263    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:18.432068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:18.432037    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2304fb8-8796-4a20-ab9c-b03fb4f34ed3" path="/var/lib/kubelet/pods/d2304fb8-8796-4a20-ab9c-b03fb4f34ed3/volumes"
Sep 08 15:30:18.445809 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:18.445775    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:18.641008 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:18.640978    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp"]
Sep 08 15:30:18.654245 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:18.654222    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d8e721-7vtwp"]
Sep 08 15:30:18.750288 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:18.750211    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:18.863285 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:18.863261    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:30:18.967672 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:18.967643    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn" podUID="6c82580c-f272-40ef-8b12-ca4d242d9e9f" containerName="main" containerID="cri-o://51edd4ebcc15ee666eedc80c952d4b9b7aded5e1a55afe1fad08091b23f05bae" gracePeriod=45
Sep 08 15:30:19.101691 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.101666    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:19.215678 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.215657    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:30:19.341931 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.341860    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:19.359188 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.359165    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-model-cache\") pod \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") "
Sep 08 15:30:19.359299 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.359209    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tmp-dir\") pod \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") "
Sep 08 15:30:19.359299 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.359228    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crk8m\" (UniqueName: \"kubernetes.io/projected/6c82580c-f272-40ef-8b12-ca4d242d9e9f-kube-api-access-crk8m\") pod \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") "
Sep 08 15:30:19.359299 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.359268    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-home\") pod \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") "
Sep 08 15:30:19.359458 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.359335    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-dshm\") pod \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") "
Sep 08 15:30:19.359458 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.359377    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs\") pod \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\" (UID: \"6c82580c-f272-40ef-8b12-ca4d242d9e9f\") "
Sep 08 15:30:19.359458 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.359451    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6c82580c-f272-40ef-8b12-ca4d242d9e9f" (UID: "6c82580c-f272-40ef-8b12-ca4d242d9e9f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:19.359610 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.359464    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-model-cache" (OuterVolumeSpecName: "model-cache") pod "6c82580c-f272-40ef-8b12-ca4d242d9e9f" (UID: "6c82580c-f272-40ef-8b12-ca4d242d9e9f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:19.359610 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.359596    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-home" (OuterVolumeSpecName: "home") pod "6c82580c-f272-40ef-8b12-ca4d242d9e9f" (UID: "6c82580c-f272-40ef-8b12-ca4d242d9e9f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:19.359767 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.359749    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:19.359829 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.359768    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:19.359829 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.359782    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:19.361467 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.361433    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6c82580c-f272-40ef-8b12-ca4d242d9e9f" (UID: "6c82580c-f272-40ef-8b12-ca4d242d9e9f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:30:19.361614 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.361443    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-dshm" (OuterVolumeSpecName: "dshm") pod "6c82580c-f272-40ef-8b12-ca4d242d9e9f" (UID: "6c82580c-f272-40ef-8b12-ca4d242d9e9f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:19.361614 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.361545    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c82580c-f272-40ef-8b12-ca4d242d9e9f-kube-api-access-crk8m" (OuterVolumeSpecName: "kube-api-access-crk8m") pod "6c82580c-f272-40ef-8b12-ca4d242d9e9f" (UID: "6c82580c-f272-40ef-8b12-ca4d242d9e9f"). InnerVolumeSpecName "kube-api-access-crk8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:30:19.461164 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.461141    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crk8m\" (UniqueName: \"kubernetes.io/projected/6c82580c-f272-40ef-8b12-ca4d242d9e9f-kube-api-access-crk8m\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:19.461164 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.461162    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c82580c-f272-40ef-8b12-ca4d242d9e9f-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:19.461291 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:19.461173    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c82580c-f272-40ef-8b12-ca4d242d9e9f-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:20.045328 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.045298    2575 generic.go:356] "Generic (PLEG): container finished" podID="6c82580c-f272-40ef-8b12-ca4d242d9e9f" containerID="51edd4ebcc15ee666eedc80c952d4b9b7aded5e1a55afe1fad08091b23f05bae" exitCode=0
Sep 08 15:30:20.045790 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.045340    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"
Sep 08 15:30:20.045790 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.045367    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn" event={"ID":"6c82580c-f272-40ef-8b12-ca4d242d9e9f","Type":"ContainerDied","Data":"51edd4ebcc15ee666eedc80c952d4b9b7aded5e1a55afe1fad08091b23f05bae"}
Sep 08 15:30:20.045790 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.045396    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn" event={"ID":"6c82580c-f272-40ef-8b12-ca4d242d9e9f","Type":"ContainerDied","Data":"9a3db94de788c0af389b3fd221529c10ac85500c4f3e2ba7494c0ed062f9045b"}
Sep 08 15:30:20.045790 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.045411    2575 scope.go:117] "RemoveContainer" containerID="51edd4ebcc15ee666eedc80c952d4b9b7aded5e1a55afe1fad08091b23f05bae"
Sep 08 15:30:20.054542 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.054520    2575 scope.go:117] "RemoveContainer" containerID="51edd4ebcc15ee666eedc80c952d4b9b7aded5e1a55afe1fad08091b23f05bae"
Sep 08 15:30:20.054815 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:20.054795    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51edd4ebcc15ee666eedc80c952d4b9b7aded5e1a55afe1fad08091b23f05bae\": container with ID starting with 51edd4ebcc15ee666eedc80c952d4b9b7aded5e1a55afe1fad08091b23f05bae not found: ID does not exist" containerID="51edd4ebcc15ee666eedc80c952d4b9b7aded5e1a55afe1fad08091b23f05bae"
Sep 08 15:30:20.054894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.054823    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51edd4ebcc15ee666eedc80c952d4b9b7aded5e1a55afe1fad08091b23f05bae"} err="failed to get container status \"51edd4ebcc15ee666eedc80c952d4b9b7aded5e1a55afe1fad08091b23f05bae\": rpc error: code = NotFound desc = could not find container \"51edd4ebcc15ee666eedc80c952d4b9b7aded5e1a55afe1fad08091b23f05bae\": container with ID starting with 51edd4ebcc15ee666eedc80c952d4b9b7aded5e1a55afe1fad08091b23f05bae not found: ID does not exist"
Sep 08 15:30:20.105254 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.105232    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"]
Sep 08 15:30:20.106996 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.106977    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-342052/noten-model-342052-kserve-567c4f4b7f-4z4hn"]
Sep 08 15:30:20.212101 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.212072    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd" podUID="aa7ea7b7-ff49-4ed4-a448-2f451dbab064" containerName="main" containerID="cri-o://b0fb50168f7d309261963cf055119213c3af2a088d000e0ad1c8740eb5ac9580" gracePeriod=45
Sep 08 15:30:20.432070 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.432040    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c82580c-f272-40ef-8b12-ca4d242d9e9f" path="/var/lib/kubelet/pods/6c82580c-f272-40ef-8b12-ca4d242d9e9f/volumes"
Sep 08 15:30:20.432610 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.432591    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dee2b102-0798-477d-80af-8ed785152dc5" path="/var/lib/kubelet/pods/dee2b102-0798-477d-80af-8ed785152dc5/volumes"
Sep 08 15:30:20.456974 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.456956    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:30:20.571247 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.571223    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-dshm\") pod \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") "
Sep 08 15:30:20.571333 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.571255    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-model-cache\") pod \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") "
Sep 08 15:30:20.571333 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.571271    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-tmp-dir\") pod \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") "
Sep 08 15:30:20.571333 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.571290    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-home\") pod \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") "
Sep 08 15:30:20.571474 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.571334    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4724\" (UniqueName: \"kubernetes.io/projected/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-kube-api-access-h4724\") pod \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") "
Sep 08 15:30:20.571474 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.571386    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-tls-certs\") pod \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\" (UID: \"aa7ea7b7-ff49-4ed4-a448-2f451dbab064\") "
Sep 08 15:30:20.571589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.571531    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-model-cache" (OuterVolumeSpecName: "model-cache") pod "aa7ea7b7-ff49-4ed4-a448-2f451dbab064" (UID: "aa7ea7b7-ff49-4ed4-a448-2f451dbab064"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:20.571589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.571546    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aa7ea7b7-ff49-4ed4-a448-2f451dbab064" (UID: "aa7ea7b7-ff49-4ed4-a448-2f451dbab064"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:20.571700 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.571610    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-home" (OuterVolumeSpecName: "home") pod "aa7ea7b7-ff49-4ed4-a448-2f451dbab064" (UID: "aa7ea7b7-ff49-4ed4-a448-2f451dbab064"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:20.571755 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.571736    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:20.571755 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.571749    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:20.571818 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.571757    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:20.573440 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.573414    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aa7ea7b7-ff49-4ed4-a448-2f451dbab064" (UID: "aa7ea7b7-ff49-4ed4-a448-2f451dbab064"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:30:20.573558 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.573463    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-dshm" (OuterVolumeSpecName: "dshm") pod "aa7ea7b7-ff49-4ed4-a448-2f451dbab064" (UID: "aa7ea7b7-ff49-4ed4-a448-2f451dbab064"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:20.573558 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.573461    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-kube-api-access-h4724" (OuterVolumeSpecName: "kube-api-access-h4724") pod "aa7ea7b7-ff49-4ed4-a448-2f451dbab064" (UID: "aa7ea7b7-ff49-4ed4-a448-2f451dbab064"). InnerVolumeSpecName "kube-api-access-h4724". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:30:20.672999 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.672979    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4724\" (UniqueName: \"kubernetes.io/projected/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-kube-api-access-h4724\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:20.672999 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.673000    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:20.673117 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:20.673009    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa7ea7b7-ff49-4ed4-a448-2f451dbab064-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:21.052539 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.052512    2575 generic.go:356] "Generic (PLEG): container finished" podID="aa7ea7b7-ff49-4ed4-a448-2f451dbab064" containerID="b0fb50168f7d309261963cf055119213c3af2a088d000e0ad1c8740eb5ac9580" exitCode=0
Sep 08 15:30:21.052881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.052542    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd" event={"ID":"aa7ea7b7-ff49-4ed4-a448-2f451dbab064","Type":"ContainerDied","Data":"b0fb50168f7d309261963cf055119213c3af2a088d000e0ad1c8740eb5ac9580"}
Sep 08 15:30:21.052881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.052555    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"
Sep 08 15:30:21.052881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.052579    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd" event={"ID":"aa7ea7b7-ff49-4ed4-a448-2f451dbab064","Type":"ContainerDied","Data":"f855fa4c002239fce8a6ba1702c1cec70c071b56084ff08d50ee9aa4ea039f78"}
Sep 08 15:30:21.052881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.052592    2575 scope.go:117] "RemoveContainer" containerID="b0fb50168f7d309261963cf055119213c3af2a088d000e0ad1c8740eb5ac9580"
Sep 08 15:30:21.061191 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.061167    2575 scope.go:117] "RemoveContainer" containerID="b0fb50168f7d309261963cf055119213c3af2a088d000e0ad1c8740eb5ac9580"
Sep 08 15:30:21.061428 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:21.061413    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0fb50168f7d309261963cf055119213c3af2a088d000e0ad1c8740eb5ac9580\": container with ID starting with b0fb50168f7d309261963cf055119213c3af2a088d000e0ad1c8740eb5ac9580 not found: ID does not exist" containerID="b0fb50168f7d309261963cf055119213c3af2a088d000e0ad1c8740eb5ac9580"
Sep 08 15:30:21.061484 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.061434    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0fb50168f7d309261963cf055119213c3af2a088d000e0ad1c8740eb5ac9580"} err="failed to get container status \"b0fb50168f7d309261963cf055119213c3af2a088d000e0ad1c8740eb5ac9580\": rpc error: code = NotFound desc = could not find container \"b0fb50168f7d309261963cf055119213c3af2a088d000e0ad1c8740eb5ac9580\": container with ID starting with b0fb50168f7d309261963cf055119213c3af2a088d000e0ad1c8740eb5ac9580 not found: ID does not exist"
Sep 08 15:30:21.127189 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.126648    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"]
Sep 08 15:30:21.132930 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.132908    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-d8e721/test-model-d8e721-kserve-776d95f4dc-lx4sd"]
Sep 08 15:30:21.389773 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.389746    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:21.421742 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.421717    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4" podUID="358950a1-db0b-481f-92cb-ca5a31c53abd" containerName="main" containerID="cri-o://071a85523a8fa8d6e6b6d60b7dbe9574c6d3b576520fbc2edd8294a38a32f60e" gracePeriod=45
Sep 08 15:30:21.660763 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.660738    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:30:21.718531 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.718502    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:21.781581 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.781557    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-tmp-dir\") pod \"358950a1-db0b-481f-92cb-ca5a31c53abd\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") "
Sep 08 15:30:21.781709 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.781600    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-model-cache\") pod \"358950a1-db0b-481f-92cb-ca5a31c53abd\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") "
Sep 08 15:30:21.781709 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.781625    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/358950a1-db0b-481f-92cb-ca5a31c53abd-tls-certs\") pod \"358950a1-db0b-481f-92cb-ca5a31c53abd\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") "
Sep 08 15:30:21.781709 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.781652    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dkzz\" (UniqueName: \"kubernetes.io/projected/358950a1-db0b-481f-92cb-ca5a31c53abd-kube-api-access-4dkzz\") pod \"358950a1-db0b-481f-92cb-ca5a31c53abd\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") "
Sep 08 15:30:21.781709 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.781684    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-dshm\") pod \"358950a1-db0b-481f-92cb-ca5a31c53abd\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") "
Sep 08 15:30:21.781926 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.781718    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-home\") pod \"358950a1-db0b-481f-92cb-ca5a31c53abd\" (UID: \"358950a1-db0b-481f-92cb-ca5a31c53abd\") "
Sep 08 15:30:21.781926 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.781842    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "358950a1-db0b-481f-92cb-ca5a31c53abd" (UID: "358950a1-db0b-481f-92cb-ca5a31c53abd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:21.781926 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.781870    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-model-cache" (OuterVolumeSpecName: "model-cache") pod "358950a1-db0b-481f-92cb-ca5a31c53abd" (UID: "358950a1-db0b-481f-92cb-ca5a31c53abd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:21.782061 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.782047    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-home" (OuterVolumeSpecName: "home") pod "358950a1-db0b-481f-92cb-ca5a31c53abd" (UID: "358950a1-db0b-481f-92cb-ca5a31c53abd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:21.782121 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.782086    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:21.782121 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.782107    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:21.783763 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.783739    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-dshm" (OuterVolumeSpecName: "dshm") pod "358950a1-db0b-481f-92cb-ca5a31c53abd" (UID: "358950a1-db0b-481f-92cb-ca5a31c53abd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:30:21.783842 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.783797    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/358950a1-db0b-481f-92cb-ca5a31c53abd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "358950a1-db0b-481f-92cb-ca5a31c53abd" (UID: "358950a1-db0b-481f-92cb-ca5a31c53abd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:30:21.783842 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.783799    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/358950a1-db0b-481f-92cb-ca5a31c53abd-kube-api-access-4dkzz" (OuterVolumeSpecName: "kube-api-access-4dkzz") pod "358950a1-db0b-481f-92cb-ca5a31c53abd" (UID: "358950a1-db0b-481f-92cb-ca5a31c53abd"). InnerVolumeSpecName "kube-api-access-4dkzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:30:21.882849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.882825    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/358950a1-db0b-481f-92cb-ca5a31c53abd-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:21.882849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.882847    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dkzz\" (UniqueName: \"kubernetes.io/projected/358950a1-db0b-481f-92cb-ca5a31c53abd-kube-api-access-4dkzz\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:21.882965 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.882856    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:21.882965 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.882865    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/358950a1-db0b-481f-92cb-ca5a31c53abd-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:21.934079 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:21.934024    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:22.059503 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:22.059462    2575 generic.go:356] "Generic (PLEG): container finished" podID="358950a1-db0b-481f-92cb-ca5a31c53abd" containerID="071a85523a8fa8d6e6b6d60b7dbe9574c6d3b576520fbc2edd8294a38a32f60e" exitCode=0
Sep 08 15:30:22.059820 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:22.059513    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"
Sep 08 15:30:22.059820 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:22.059526    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4" event={"ID":"358950a1-db0b-481f-92cb-ca5a31c53abd","Type":"ContainerDied","Data":"071a85523a8fa8d6e6b6d60b7dbe9574c6d3b576520fbc2edd8294a38a32f60e"}
Sep 08 15:30:22.059820 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:22.059562    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4" event={"ID":"358950a1-db0b-481f-92cb-ca5a31c53abd","Type":"ContainerDied","Data":"3ab784a38436a4eb412784bc3a66d7f6ac3373541739341dbd741e515dd2ce63"}
Sep 08 15:30:22.059820 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:22.059582    2575 scope.go:117] "RemoveContainer" containerID="071a85523a8fa8d6e6b6d60b7dbe9574c6d3b576520fbc2edd8294a38a32f60e"
Sep 08 15:30:22.068601 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:22.068580    2575 scope.go:117] "RemoveContainer" containerID="071a85523a8fa8d6e6b6d60b7dbe9574c6d3b576520fbc2edd8294a38a32f60e"
Sep 08 15:30:22.068861 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:30:22.068843    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"071a85523a8fa8d6e6b6d60b7dbe9574c6d3b576520fbc2edd8294a38a32f60e\": container with ID starting with 071a85523a8fa8d6e6b6d60b7dbe9574c6d3b576520fbc2edd8294a38a32f60e not found: ID does not exist" containerID="071a85523a8fa8d6e6b6d60b7dbe9574c6d3b576520fbc2edd8294a38a32f60e"
Sep 08 15:30:22.068916 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:22.068868    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"071a85523a8fa8d6e6b6d60b7dbe9574c6d3b576520fbc2edd8294a38a32f60e"} err="failed to get container status \"071a85523a8fa8d6e6b6d60b7dbe9574c6d3b576520fbc2edd8294a38a32f60e\": rpc error: code = NotFound desc = could not find container \"071a85523a8fa8d6e6b6d60b7dbe9574c6d3b576520fbc2edd8294a38a32f60e\": container with ID starting with 071a85523a8fa8d6e6b6d60b7dbe9574c6d3b576520fbc2edd8294a38a32f60e not found: ID does not exist"
Sep 08 15:30:22.130470 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:22.130446    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"]
Sep 08 15:30:22.170868 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:22.170849    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-677a6c/test-model-677a6c-kserve-5478b95d45-j8wb4"]
Sep 08 15:30:22.431562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:22.431536    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="358950a1-db0b-481f-92cb-ca5a31c53abd" path="/var/lib/kubelet/pods/358950a1-db0b-481f-92cb-ca5a31c53abd/volumes"
Sep 08 15:30:22.431897 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:22.431885    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa7ea7b7-ff49-4ed4-a448-2f451dbab064" path="/var/lib/kubelet/pods/aa7ea7b7-ff49-4ed4-a448-2f451dbab064/volumes"
Sep 08 15:30:22.576843 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:22.576816    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:22.785003 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:22.784944    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:24.559863 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:24.559830    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:26.530669 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:26.530639    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:30:28.902202 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:28.902168    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:29.080028 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.080001    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:29.326929 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.326841    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:29.724326 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724296    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87"]
Sep 08 15:30:29.724737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724720    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa7ea7b7-ff49-4ed4-a448-2f451dbab064" containerName="main"
Sep 08 15:30:29.724839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724739    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa7ea7b7-ff49-4ed4-a448-2f451dbab064" containerName="main"
Sep 08 15:30:29.724839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724767    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dee2b102-0798-477d-80af-8ed785152dc5" containerName="revoke-keys"
Sep 08 15:30:29.724839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724778    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dee2b102-0798-477d-80af-8ed785152dc5" containerName="revoke-keys"
Sep 08 15:30:29.724839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724791    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="358950a1-db0b-481f-92cb-ca5a31c53abd" containerName="main"
Sep 08 15:30:29.724839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724800    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="358950a1-db0b-481f-92cb-ca5a31c53abd" containerName="main"
Sep 08 15:30:29.724839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724812    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c82580c-f272-40ef-8b12-ca4d242d9e9f" containerName="main"
Sep 08 15:30:29.724839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724820    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c82580c-f272-40ef-8b12-ca4d242d9e9f" containerName="main"
Sep 08 15:30:29.725161 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724845    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29903f8-e0c2-444f-a194-5326761d54d0" containerName="istio-proxy"
Sep 08 15:30:29.725161 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724853    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29903f8-e0c2-444f-a194-5326761d54d0" containerName="istio-proxy"
Sep 08 15:30:29.725161 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724937    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="358950a1-db0b-481f-92cb-ca5a31c53abd" containerName="main"
Sep 08 15:30:29.725161 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724952    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c29903f8-e0c2-444f-a194-5326761d54d0" containerName="istio-proxy"
Sep 08 15:30:29.725161 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724964    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c82580c-f272-40ef-8b12-ca4d242d9e9f" containerName="main"
Sep 08 15:30:29.725161 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724972    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dee2b102-0798-477d-80af-8ed785152dc5" containerName="revoke-keys"
Sep 08 15:30:29.725161 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.724982    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa7ea7b7-ff49-4ed4-a448-2f451dbab064" containerName="main"
Sep 08 15:30:29.729438 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.729420    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87"
Sep 08 15:30:29.734301 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.734283    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:30:29.734409 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.734354    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8qqvg\""
Sep 08 15:30:29.845152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.845128    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87"]
Sep 08 15:30:29.845328 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.845307    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1694fa8-5ffc-415b-81e8-789e7f977383-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-677a6c-48q87\" (UID: \"b1694fa8-5ffc-415b-81e8-789e7f977383\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87"
Sep 08 15:30:29.901533 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.901484    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:29.946619 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.946594    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1694fa8-5ffc-415b-81e8-789e7f977383-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-677a6c-48q87\" (UID: \"b1694fa8-5ffc-415b-81e8-789e7f977383\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87"
Sep 08 15:30:29.947160 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:29.947142    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1694fa8-5ffc-415b-81e8-789e7f977383-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-677a6c-48q87\" (UID: \"b1694fa8-5ffc-415b-81e8-789e7f977383\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87"
Sep 08 15:30:30.039302 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:30.039242    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87"
Sep 08 15:30:30.053299 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:30.053276    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:30.493442 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:30:30.493412    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1694fa8_5ffc_415b_81e8_789e7f977383.slice/crio-2026811087ef863382ff8af054a92049d0d679bccc597a2ae0b788e80dba3302 WatchSource:0}: Error finding container 2026811087ef863382ff8af054a92049d0d679bccc597a2ae0b788e80dba3302: Status 404 returned error can't find the container with id 2026811087ef863382ff8af054a92049d0d679bccc597a2ae0b788e80dba3302
Sep 08 15:30:30.538475 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:30.538446    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87"]
Sep 08 15:30:31.111644 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:31.111611    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87" event={"ID":"b1694fa8-5ffc-415b-81e8-789e7f977383","Type":"ContainerStarted","Data":"d126dd518f5aa5cef218f1a4a1069b19dae4fa60d99dc9905eeb98d922efb592"}
Sep 08 15:30:31.111971 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:31.111663    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87" event={"ID":"b1694fa8-5ffc-415b-81e8-789e7f977383","Type":"ContainerStarted","Data":"2026811087ef863382ff8af054a92049d0d679bccc597a2ae0b788e80dba3302"}
Sep 08 15:30:31.191979 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:31.191931    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87" podStartSLOduration=2.191917594 podStartE2EDuration="2.191917594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:31.189681347 +0000 UTC m=+1279.275631397" watchObservedRunningTime="2026-09-08 15:30:31.191917594 +0000 UTC m=+1279.277867631"
Sep 08 15:30:32.118947 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:32.118914    2575 generic.go:356] "Generic (PLEG): container finished" podID="b1694fa8-5ffc-415b-81e8-789e7f977383" containerID="d126dd518f5aa5cef218f1a4a1069b19dae4fa60d99dc9905eeb98d922efb592" exitCode=0
Sep 08 15:30:32.119317 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:32.118996    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87" event={"ID":"b1694fa8-5ffc-415b-81e8-789e7f977383","Type":"ContainerDied","Data":"d126dd518f5aa5cef218f1a4a1069b19dae4fa60d99dc9905eeb98d922efb592"}
Sep 08 15:30:33.248473 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:33.248445    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87"
Sep 08 15:30:33.275070 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:33.275044    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1694fa8-5ffc-415b-81e8-789e7f977383-maas-api-service-ca\") pod \"b1694fa8-5ffc-415b-81e8-789e7f977383\" (UID: \"b1694fa8-5ffc-415b-81e8-789e7f977383\") "
Sep 08 15:30:33.275530 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:33.275477    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1694fa8-5ffc-415b-81e8-789e7f977383-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b1694fa8-5ffc-415b-81e8-789e7f977383" (UID: "b1694fa8-5ffc-415b-81e8-789e7f977383"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:30:33.275866 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:33.275836    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1694fa8-5ffc-415b-81e8-789e7f977383-maas-api-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:30:34.130699 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:34.130666    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87"
Sep 08 15:30:34.130881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:34.130691    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87" event={"ID":"b1694fa8-5ffc-415b-81e8-789e7f977383","Type":"ContainerDied","Data":"2026811087ef863382ff8af054a92049d0d679bccc597a2ae0b788e80dba3302"}
Sep 08 15:30:34.130881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:34.130776    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2026811087ef863382ff8af054a92049d0d679bccc597a2ae0b788e80dba3302"
Sep 08 15:30:34.527520 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:34.527427    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:34.828639 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:34.828552    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:35.333308 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:35.333275    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:37.403545 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:37.403515    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:30:40.882849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:40.882818    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc"]
Sep 08 15:30:40.893141 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:40.893112    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-677a6c-29814690-rndvc"]
Sep 08 15:30:40.932245 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:40.932209    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:41.224407 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:41.224338    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:41.547017 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:41.546939    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:41.609871 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:41.609842    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87"]
Sep 08 15:30:41.614569 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:41.614547    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-677a6c-48q87"]
Sep 08 15:30:41.814562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:41.814470    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:42.137993 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:42.137937    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:42.432152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:42.432076    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1694fa8-5ffc-415b-81e8-789e7f977383" path="/var/lib/kubelet/pods/b1694fa8-5ffc-415b-81e8-789e7f977383/volumes"
Sep 08 15:30:42.432366 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:42.432354    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dba24b3c-5002-458d-b2f8-827b204368a1" path="/var/lib/kubelet/pods/dba24b3c-5002-458d-b2f8-827b204368a1/volumes"
Sep 08 15:30:45.496827 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:45.496789    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:45.656701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:45.656656    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:45.823416 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:45.823331    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:46.135383 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.135349    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"]
Sep 08 15:30:46.135741 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.135728    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1694fa8-5ffc-415b-81e8-789e7f977383" containerName="revoke-keys"
Sep 08 15:30:46.135796 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.135742    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1694fa8-5ffc-415b-81e8-789e7f977383" containerName="revoke-keys"
Sep 08 15:30:46.135838 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.135808    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1694fa8-5ffc-415b-81e8-789e7f977383" containerName="revoke-keys"
Sep 08 15:30:46.138747 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.138730    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.146948 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.146927    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-49bfbc48-openshift-default-dockercfg-wtpj6\""
Sep 08 15:30:46.165755 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.165732    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"]
Sep 08 15:30:46.186008 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.185983    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-credential-socket\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.186103 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.186017    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-workload-certs\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.186103 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.186038    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-token\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.186173 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.186140    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-data\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.186259 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.186230    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-podinfo\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.186339 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.186322    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-workload-socket\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.186374 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.186366    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aec05402-1b16-4942-b4e4-7d28c306c0b8-istiod-ca-cert\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.186429 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.186416    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-envoy\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.186464 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.186446    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqb7m\" (UniqueName: \"kubernetes.io/projected/aec05402-1b16-4942-b4e4-7d28c306c0b8-kube-api-access-wqb7m\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.200235 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.200211    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:46.286943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.286911    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-workload-socket\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.287142 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.286952    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aec05402-1b16-4942-b4e4-7d28c306c0b8-istiod-ca-cert\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.287142 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.286976    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-envoy\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.287142 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.286994    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqb7m\" (UniqueName: \"kubernetes.io/projected/aec05402-1b16-4942-b4e4-7d28c306c0b8-kube-api-access-wqb7m\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.287142 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.287028    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-credential-socket\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.287142 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.287047    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-workload-certs\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.287142 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.287071    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-token\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.287142 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.287133    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-data\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.287535 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.287172    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-podinfo\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.287535 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.287281    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-workload-socket\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.287535 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.287437    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-credential-socket\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.287708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.287682    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-data\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.287708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.287694    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-workload-certs\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.287793 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.287770    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aec05402-1b16-4942-b4e4-7d28c306c0b8-istiod-ca-cert\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.289253 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.289234    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-envoy\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.289459 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.289442    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-podinfo\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.317765 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.317735    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqb7m\" (UniqueName: \"kubernetes.io/projected/aec05402-1b16-4942-b4e4-7d28c306c0b8-kube-api-access-wqb7m\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.318039 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.318019    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-token\") pod \"e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") " pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.419248 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.419160    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:46.448533 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.448486    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:46.765673 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.765648    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"]
Sep 08 15:30:46.766954 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:30:46.766928    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec05402_1b16_4942_b4e4_7d28c306c0b8.slice/crio-915f35d37f80877b5cb9a6e2fcbc1162cd6c508dc4b037b24608afcbfe1729f9 WatchSource:0}: Error finding container 915f35d37f80877b5cb9a6e2fcbc1162cd6c508dc4b037b24608afcbfe1729f9: Status 404 returned error can't find the container with id 915f35d37f80877b5cb9a6e2fcbc1162cd6c508dc4b037b24608afcbfe1729f9
Sep 08 15:30:46.768953 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.768924    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:30:46.769034 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.768983    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:30:46.769034 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:46.769009    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:30:47.196443 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:47.196406    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" event={"ID":"aec05402-1b16-4942-b4e4-7d28c306c0b8","Type":"ContainerStarted","Data":"2e7cf41757e7fa112e6a3d406297feea62a4d63d62c9846f0f4f5b6a0ff0a1bd"}
Sep 08 15:30:47.196638 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:47.196464    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" event={"ID":"aec05402-1b16-4942-b4e4-7d28c306c0b8","Type":"ContainerStarted","Data":"915f35d37f80877b5cb9a6e2fcbc1162cd6c508dc4b037b24608afcbfe1729f9"}
Sep 08 15:30:47.291338 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:47.291273    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" podStartSLOduration=1.291243803 podStartE2EDuration="1.291243803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:47.288748104 +0000 UTC m=+1295.374698157" watchObservedRunningTime="2026-09-08 15:30:47.291243803 +0000 UTC m=+1295.377193846"
Sep 08 15:30:47.449687 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:47.449592    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:47.459517 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:47.454866    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:48.173346 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:48.173314    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:30:48.202004 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:48.201978    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:48.202901 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:48.202880    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:30:49.483911 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:49.483877    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:49.696414 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:49.696380    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:49.915890 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:49.915855    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:51.298469 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.298437    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"]
Sep 08 15:30:51.302007 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.301987    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.347779 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.347755    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:30:51.347909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.347794    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-49bfbc48-serving-cert\""
Sep 08 15:30:51.353267 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.353252    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hrbm8\""
Sep 08 15:30:51.365767 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.365744    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 08 15:30:51.366128 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.366106    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"]
Sep 08 15:30:51.378403 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.378376    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 08 15:30:51.383048 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.383027    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:51.433676 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.433651    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccefc034-b06a-451b-ba73-db9898ee8aed-openshift-service-ca\") pod \"maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.433786 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.433685    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccefc034-b06a-451b-ba73-db9898ee8aed-maas-api-tls\") pod \"maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.433786 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.433720    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqgdm\" (UniqueName: \"kubernetes.io/projected/ccefc034-b06a-451b-ba73-db9898ee8aed-kube-api-access-zqgdm\") pod \"maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.433873 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.433806    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccefc034-b06a-451b-ba73-db9898ee8aed-metrics-certs\") pod \"maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.535117 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.535082    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqgdm\" (UniqueName: \"kubernetes.io/projected/ccefc034-b06a-451b-ba73-db9898ee8aed-kube-api-access-zqgdm\") pod \"maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.535292 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.535128    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccefc034-b06a-451b-ba73-db9898ee8aed-metrics-certs\") pod \"maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.535292 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.535198    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccefc034-b06a-451b-ba73-db9898ee8aed-openshift-service-ca\") pod \"maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.535292 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.535221    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccefc034-b06a-451b-ba73-db9898ee8aed-maas-api-tls\") pod \"maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.535974 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.535949    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccefc034-b06a-451b-ba73-db9898ee8aed-openshift-service-ca\") pod \"maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.537627 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.537607    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccefc034-b06a-451b-ba73-db9898ee8aed-maas-api-tls\") pod \"maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.537705 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.537652    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccefc034-b06a-451b-ba73-db9898ee8aed-metrics-certs\") pod \"maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.572900 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.572842    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqgdm\" (UniqueName: \"kubernetes.io/projected/ccefc034-b06a-451b-ba73-db9898ee8aed-kube-api-access-zqgdm\") pod \"maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.612440 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.612410    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:51.827523 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.827413    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:51.960069 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.960027    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"]
Sep 08 15:30:51.963934 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:30:51.963909    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccefc034_b06a_451b_ba73_db9898ee8aed.slice/crio-996738924d7019d5449f9f518a45cf989ea880f80462d959b33806e2e4db9f09 WatchSource:0}: Error finding container 996738924d7019d5449f9f518a45cf989ea880f80462d959b33806e2e4db9f09: Status 404 returned error can't find the container with id 996738924d7019d5449f9f518a45cf989ea880f80462d959b33806e2e4db9f09
Sep 08 15:30:51.965269 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:51.965254    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:30:52.133552 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:52.133517    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:52.222051 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:52.222010    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq" event={"ID":"ccefc034-b06a-451b-ba73-db9898ee8aed","Type":"ContainerStarted","Data":"996738924d7019d5449f9f518a45cf989ea880f80462d959b33806e2e4db9f09"}
Sep 08 15:30:52.890343 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:52.890312    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:53.193058 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:53.192975    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:54.233407 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:54.233374    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq" event={"ID":"ccefc034-b06a-451b-ba73-db9898ee8aed","Type":"ContainerStarted","Data":"6a70b0f1fd525c14a96ee84a66bb4e5de7e775e26f8e3ad360e0e9db3a073a9a"}
Sep 08 15:30:54.233792 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:54.233612    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:30:54.280323 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:54.280274    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq" podStartSLOduration=1.57740422 podStartE2EDuration="3.280258768s" totalImagesPullingTime="1.702854548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:54.275506802 +0000 UTC m=+1302.361456844" watchObservedRunningTime="2026-09-08 15:30:54.280258768 +0000 UTC m=+1302.366208807"
Sep 08 15:30:54.317887 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:54.317857    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:54.464774 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:54.464735    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:54.657923 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:54.657884    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:55.222110 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.222073    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:55.636120 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.636083    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"]
Sep 08 15:30:55.639652 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.639635    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.643728 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.643711    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-2e082e-openshift-default-dockercfg-qdmjv\""
Sep 08 15:30:55.703356 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.703328    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"]
Sep 08 15:30:55.771775 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.771745    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:55.773917 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.773894    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1761f785-e4f8-4021-a91c-0d4ae9421f88-istiod-ca-cert\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.773994 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.773942    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-data\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.774058 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.774038    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-token\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.774113 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.774073    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-podinfo\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.774180 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.774163    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-workload-certs\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.774232 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.774216    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-credential-socket\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.774290 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.774260    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdnv5\" (UniqueName: \"kubernetes.io/projected/1761f785-e4f8-4021-a91c-0d4ae9421f88-kube-api-access-hdnv5\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.774334 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.774287    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-envoy\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.774334 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.774320    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-workload-socket\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.875924 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.875892    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-workload-certs\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.876075 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.875951    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-credential-socket\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.876075 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.875979    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdnv5\" (UniqueName: \"kubernetes.io/projected/1761f785-e4f8-4021-a91c-0d4ae9421f88-kube-api-access-hdnv5\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.876075 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.876006    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-envoy\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.876075 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.876034    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-workload-socket\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.876075 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.876074    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1761f785-e4f8-4021-a91c-0d4ae9421f88-istiod-ca-cert\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.876343 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.876126    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-data\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.876343 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.876175    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-token\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.876343 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.876211    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-podinfo\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.876465 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.876344    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-workload-certs\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.876465 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.876380    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-credential-socket\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.876582 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.876566    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-workload-socket\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.876621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.876603    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-data\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.876868 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.876837    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1761f785-e4f8-4021-a91c-0d4ae9421f88-istiod-ca-cert\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.878505 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.878467    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-envoy\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.878589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.878546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-podinfo\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.888373 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.888314    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-token\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.888544 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.888528    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdnv5\" (UniqueName: \"kubernetes.io/projected/1761f785-e4f8-4021-a91c-0d4ae9421f88-kube-api-access-hdnv5\") pod \"e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") " pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:55.949226 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:55.949178    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:56.026684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:56.026639    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:56.231692 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:56.231667    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"]
Sep 08 15:30:56.233468 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:30:56.233444    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1761f785_e4f8_4021_a91c_0d4ae9421f88.slice/crio-23375f357a7f35cb6b2a7360e5a101abae5dc70c3c47f5395b1ee0451be2ffff WatchSource:0}: Error finding container 23375f357a7f35cb6b2a7360e5a101abae5dc70c3c47f5395b1ee0451be2ffff: Status 404 returned error can't find the container with id 23375f357a7f35cb6b2a7360e5a101abae5dc70c3c47f5395b1ee0451be2ffff
Sep 08 15:30:56.235551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:56.235521    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:30:56.235616 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:56.235584    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:30:56.235616 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:56.235610    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:30:56.244935 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:56.244910    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" event={"ID":"1761f785-e4f8-4021-a91c-0d4ae9421f88","Type":"ContainerStarted","Data":"23375f357a7f35cb6b2a7360e5a101abae5dc70c3c47f5395b1ee0451be2ffff"}
Sep 08 15:30:57.251883 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:57.251847    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" event={"ID":"1761f785-e4f8-4021-a91c-0d4ae9421f88","Type":"ContainerStarted","Data":"89ae883fa70679ded8578ee0154fe1d0773a187d027b79060ee6580efa1b62b1"}
Sep 08 15:30:57.288517 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:57.288455    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" podStartSLOduration=2.288439998 podStartE2EDuration="2.288439998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:30:57.285770001 +0000 UTC m=+1305.371720041" watchObservedRunningTime="2026-09-08 15:30:57.288439998 +0000 UTC m=+1305.374390037"
Sep 08 15:30:57.949307 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:57.949270    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:57.954228 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:57.954200    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:58.257601 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:58.257481    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:58.258605 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:58.258587    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:30:59.445069 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:59.445036    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:59.553408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:59.553354    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:30:59.660364 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:30:59.660327    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:31:00.244332 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:00.244303    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:31:00.751739 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:00.751688    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:00.856345 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:00.856312    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:01.154014 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:01.153980    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:01.506166 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:01.506063    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:01.658944 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:01.658898    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:04.243662 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:04.243617    2575 patch_prober.go:28] interesting pod/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 15:31:04.244072 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:04.243681    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 15:31:05.296824 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:05.296794    2575 generic.go:356] "Generic (PLEG): container finished" podID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerID="2e7cf41757e7fa112e6a3d406297feea62a4d63d62c9846f0f4f5b6a0ff0a1bd" exitCode=0
Sep 08 15:31:05.297197 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:05.296865    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" event={"ID":"aec05402-1b16-4942-b4e4-7d28c306c0b8","Type":"ContainerDied","Data":"2e7cf41757e7fa112e6a3d406297feea62a4d63d62c9846f0f4f5b6a0ff0a1bd"}
Sep 08 15:31:05.297261 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:05.297246    2575 scope.go:117] "RemoveContainer" containerID="2e7cf41757e7fa112e6a3d406297feea62a4d63d62c9846f0f4f5b6a0ff0a1bd"
Sep 08 15:31:05.298213 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:05.298191    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:31:05.298290 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:05.298254    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:31:05.298290 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:05.298287    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:31:06.307606 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:06.307566    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" event={"ID":"aec05402-1b16-4942-b4e4-7d28c306c0b8","Type":"ContainerStarted","Data":"3a2d8b1fd04cbce71d1c9b0ba90b982fe9b5a3345c0807c0df67e6d924df105a"}
Sep 08 15:31:06.449135 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:06.449099    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:31:06.449426 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:06.449405    2575 patch_prober.go:28] interesting pod/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 08 15:31:06.449475 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:06.449452    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 08 15:31:07.449149 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:07.448810    2575 patch_prober.go:28] interesting pod/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 08 15:31:07.449149 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:07.448863    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 08 15:31:08.449077 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:08.449040    2575 patch_prober.go:28] interesting pod/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 08 15:31:08.449251 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:08.449089    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 08 15:31:09.448823 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:09.448781    2575 patch_prober.go:28] interesting pod/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 08 15:31:09.449024 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:09.448850    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 08 15:31:10.274566 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:10.274535    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:31:10.449479 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:10.449446    2575 patch_prober.go:28] interesting pod/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 08 15:31:10.449652 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:10.449531    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 08 15:31:11.449253 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:11.449212    2575 patch_prober.go:28] interesting pod/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 08 15:31:11.449668 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:11.449285    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 08 15:31:12.452930 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:12.452900    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:31:12.453304 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:12.453143    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:31:12.453921 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:12.453903    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:31:13.970309 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:13.969839    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:33936->10.134.0.63:15021: read: connection reset by peer" start-of-body=
Sep 08 15:31:13.970309 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:13.969904    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:33936->10.134.0.63:15021: read: connection reset by peer"
Sep 08 15:31:14.379296 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:14.379267    2575 generic.go:356] "Generic (PLEG): container finished" podID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerID="89ae883fa70679ded8578ee0154fe1d0773a187d027b79060ee6580efa1b62b1" exitCode=0
Sep 08 15:31:14.379441 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:14.379338    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" event={"ID":"1761f785-e4f8-4021-a91c-0d4ae9421f88","Type":"ContainerDied","Data":"89ae883fa70679ded8578ee0154fe1d0773a187d027b79060ee6580efa1b62b1"}
Sep 08 15:31:14.379823 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:14.379805    2575 scope.go:117] "RemoveContainer" containerID="89ae883fa70679ded8578ee0154fe1d0773a187d027b79060ee6580efa1b62b1"
Sep 08 15:31:14.380803 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:14.380776    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:31:14.380873 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:14.380855    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:31:14.380913 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:14.380892    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:31:15.385878 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:15.385841    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" event={"ID":"1761f785-e4f8-4021-a91c-0d4ae9421f88","Type":"ContainerStarted","Data":"565fe40be38991611bcf81c4c98a4759a7712834f19801e3d23fdf72e5d2f3d8"}
Sep 08 15:31:15.462761 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:15.462725    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:15.950356 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:15.950322    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:31:15.950601 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:15.950578    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 08 15:31:15.950652 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:15.950636    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 08 15:31:16.949684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:16.949652    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 08 15:31:16.950091 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:16.949706    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 08 15:31:17.949884 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:17.949839    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 08 15:31:17.950277 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:17.949907    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 08 15:31:18.950029 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:18.949990    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 08 15:31:18.950434 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:18.950046    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 08 15:31:19.606434 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:19.606386    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:19.950368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:19.950285    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 08 15:31:19.950368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:19.950340    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 08 15:31:20.374935 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:20.374903    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:20.502908 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:20.502881    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:20.852657 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:20.852631    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:20.953170 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:20.953146    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:31:20.953608 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:20.953515    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:31:20.954144 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:20.954124    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:31:22.253889 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:22.253852    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:24.319016 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:24.318982    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:24.688097 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:24.688065    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:25.114232 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:25.114195    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:25.219388 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:25.219355    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:27.824738 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:27.824706    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:27.971846 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:27.971819    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:28.080468 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:28.080393    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:28.179079 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:28.179040    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:28.457256 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:28.457222    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl"]
Sep 08 15:31:28.460285 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:28.460263    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl"
Sep 08 15:31:28.463228 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:28.463211    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8qqvg\""
Sep 08 15:31:28.469165 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:28.469138    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl"]
Sep 08 15:31:28.512955 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:28.512931    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3795a140-a911-40ae-a195-b93fb48a3fd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl\" (UID: \"3795a140-a911-40ae-a195-b93fb48a3fd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl"
Sep 08 15:31:28.573004 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:28.572975    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:28.614105 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:28.614080    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3795a140-a911-40ae-a195-b93fb48a3fd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl\" (UID: \"3795a140-a911-40ae-a195-b93fb48a3fd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl"
Sep 08 15:31:28.614688 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:28.614669    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3795a140-a911-40ae-a195-b93fb48a3fd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl\" (UID: \"3795a140-a911-40ae-a195-b93fb48a3fd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl"
Sep 08 15:31:28.772195 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:28.772118    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl"
Sep 08 15:31:29.053237 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:31:29.053210    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3795a140_a911_40ae_a195_b93fb48a3fd4.slice/crio-f831a1ef8db5ceafd5dab0b1a4ede0516bc3583480b268216651f47d19947943 WatchSource:0}: Error finding container f831a1ef8db5ceafd5dab0b1a4ede0516bc3583480b268216651f47d19947943: Status 404 returned error can't find the container with id f831a1ef8db5ceafd5dab0b1a4ede0516bc3583480b268216651f47d19947943
Sep 08 15:31:29.053662 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.053569    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl"]
Sep 08 15:31:29.465927 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.465894    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl" event={"ID":"3795a140-a911-40ae-a195-b93fb48a3fd4","Type":"ContainerStarted","Data":"1d291cc784731b104a12c64dbf65d847df88fb8cf2792a342a91c38dfb11a700"}
Sep 08 15:31:29.466098 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.465945    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl" event={"ID":"3795a140-a911-40ae-a195-b93fb48a3fd4","Type":"ContainerStarted","Data":"f831a1ef8db5ceafd5dab0b1a4ede0516bc3583480b268216651f47d19947943"}
Sep 08 15:31:29.489077 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.489027    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl" podStartSLOduration=1.489011583 podStartE2EDuration="1.489011583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:31:29.487082309 +0000 UTC m=+1337.573032351" watchObservedRunningTime="2026-09-08 15:31:29.489011583 +0000 UTC m=+1337.574961624"
Sep 08 15:31:29.572769 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.572689    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"]
Sep 08 15:31:29.575594 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.575568    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:29.580801 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.580780    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-bc9cdf-serving-cert\""
Sep 08 15:31:29.587660 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.587636    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"]
Sep 08 15:31:29.621137 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.621106    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:29.624273 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.624243    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:29.624391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.624326    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92a8fe-14f5-4bf0-867b-fe6114321b07-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:29.624391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.624380    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:29.624468 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.624402    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dqvv\" (UniqueName: \"kubernetes.io/projected/4a92a8fe-14f5-4bf0-867b-fe6114321b07-kube-api-access-6dqvv\") pod \"maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:29.720875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.720835    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:29.724995 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.724974    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:29.725052 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.725035    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92a8fe-14f5-4bf0-867b-fe6114321b07-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:29.725099 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.725083    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:29.725136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.725106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dqvv\" (UniqueName: \"kubernetes.io/projected/4a92a8fe-14f5-4bf0-867b-fe6114321b07-kube-api-access-6dqvv\") pod \"maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:29.725136 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:31:29.725113    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-serving-cert: secret "maas-api-e2e-shared-b-w3-bc9cdf-serving-cert" not found
Sep 08 15:31:29.725202 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:31:29.725174    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-maas-api-tls podName:4a92a8fe-14f5-4bf0-867b-fe6114321b07 nodeName:}" failed. No retries permitted until 2026-09-08 15:31:30.225154866 +0000 UTC m=+1338.311104887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-maas-api-tls") pod "maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg" (UID: "4a92a8fe-14f5-4bf0-867b-fe6114321b07") : secret "maas-api-e2e-shared-b-w3-bc9cdf-serving-cert" not found
Sep 08 15:31:29.725701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.725682    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92a8fe-14f5-4bf0-867b-fe6114321b07-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:29.727391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.727376    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:29.738276 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.738253    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dqvv\" (UniqueName: \"kubernetes.io/projected/4a92a8fe-14f5-4bf0-867b-fe6114321b07-kube-api-access-6dqvv\") pod \"maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:29.828484 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:29.828430    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:30.230018 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:30.229983    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:30.232356 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:30.232331    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:30.265865 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:30.265838    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:30.395793 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:30.395763    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:30.471479 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:30.471455    2575 generic.go:356] "Generic (PLEG): container finished" podID="3795a140-a911-40ae-a195-b93fb48a3fd4" containerID="1d291cc784731b104a12c64dbf65d847df88fb8cf2792a342a91c38dfb11a700" exitCode=0
Sep 08 15:31:30.471608 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:30.471542    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl" event={"ID":"3795a140-a911-40ae-a195-b93fb48a3fd4","Type":"ContainerDied","Data":"1d291cc784731b104a12c64dbf65d847df88fb8cf2792a342a91c38dfb11a700"}
Sep 08 15:31:30.486517 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:30.486446    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:30.811787 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:30.811764    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"]
Sep 08 15:31:30.813989 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:31:30.813961    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a92a8fe_14f5_4bf0_867b_fe6114321b07.slice/crio-841f3f01f9239e15b5d94f13217706fb73dfb605d09d4f2452b1a11edf73c955 WatchSource:0}: Error finding container 841f3f01f9239e15b5d94f13217706fb73dfb605d09d4f2452b1a11edf73c955: Status 404 returned error can't find the container with id 841f3f01f9239e15b5d94f13217706fb73dfb605d09d4f2452b1a11edf73c955
Sep 08 15:31:30.997367 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:30.997339    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:31.478436 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:31.478344    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg" event={"ID":"4a92a8fe-14f5-4bf0-867b-fe6114321b07","Type":"ContainerStarted","Data":"29def99d830c0f7a02be834dad3d31bd54d8169461ed53976de5b12af1242494"}
Sep 08 15:31:31.478436 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:31.478400    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg" event={"ID":"4a92a8fe-14f5-4bf0-867b-fe6114321b07","Type":"ContainerStarted","Data":"841f3f01f9239e15b5d94f13217706fb73dfb605d09d4f2452b1a11edf73c955"}
Sep 08 15:31:31.478865 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:31.478560    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:31.506844 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:31.506796    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg" podStartSLOduration=2.181295715 podStartE2EDuration="2.506779396s" totalImagesPullingTime="325.483681ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:31:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:31:31.505699354 +0000 UTC m=+1339.591649396" watchObservedRunningTime="2026-09-08 15:31:31.506779396 +0000 UTC m=+1339.592729438"
Sep 08 15:31:31.610580 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:31.610559    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl"
Sep 08 15:31:31.745671 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:31.745590    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3795a140-a911-40ae-a195-b93fb48a3fd4-maas-api-service-ca\") pod \"3795a140-a911-40ae-a195-b93fb48a3fd4\" (UID: \"3795a140-a911-40ae-a195-b93fb48a3fd4\") "
Sep 08 15:31:31.745911 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:31.745890    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3795a140-a911-40ae-a195-b93fb48a3fd4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3795a140-a911-40ae-a195-b93fb48a3fd4" (UID: "3795a140-a911-40ae-a195-b93fb48a3fd4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:31:31.847290 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:31.847262    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3795a140-a911-40ae-a195-b93fb48a3fd4-maas-api-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:32.113729 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:32.113699    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:32.484769 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:32.484740    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl"
Sep 08 15:31:32.485082 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:32.484788    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl" event={"ID":"3795a140-a911-40ae-a195-b93fb48a3fd4","Type":"ContainerDied","Data":"f831a1ef8db5ceafd5dab0b1a4ede0516bc3583480b268216651f47d19947943"}
Sep 08 15:31:32.485082 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:32.484830    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f831a1ef8db5ceafd5dab0b1a4ede0516bc3583480b268216651f47d19947943"
Sep 08 15:31:34.789233 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:34.789150    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:31:37.490257 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:37.490230    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:31:38.671510 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:38.671456    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:39.412677 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.412642    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"]
Sep 08 15:31:39.412921 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.412873    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq" podUID="ccefc034-b06a-451b-ba73-db9898ee8aed" containerName="maas-api" containerID="cri-o://6a70b0f1fd525c14a96ee84a66bb4e5de7e775e26f8e3ad360e0e9db3a073a9a" gracePeriod=30
Sep 08 15:31:39.479888 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.479856    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:39.605462 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.605427    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:39.659521 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.659481    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:31:39.783468 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.783379    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:39.821432 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.821402    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccefc034-b06a-451b-ba73-db9898ee8aed-metrics-certs\") pod \"ccefc034-b06a-451b-ba73-db9898ee8aed\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") "
Sep 08 15:31:39.821628 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.821437    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccefc034-b06a-451b-ba73-db9898ee8aed-openshift-service-ca\") pod \"ccefc034-b06a-451b-ba73-db9898ee8aed\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") "
Sep 08 15:31:39.821628 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.821456    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqgdm\" (UniqueName: \"kubernetes.io/projected/ccefc034-b06a-451b-ba73-db9898ee8aed-kube-api-access-zqgdm\") pod \"ccefc034-b06a-451b-ba73-db9898ee8aed\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") "
Sep 08 15:31:39.821628 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.821477    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccefc034-b06a-451b-ba73-db9898ee8aed-maas-api-tls\") pod \"ccefc034-b06a-451b-ba73-db9898ee8aed\" (UID: \"ccefc034-b06a-451b-ba73-db9898ee8aed\") "
Sep 08 15:31:39.821902 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.821875    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccefc034-b06a-451b-ba73-db9898ee8aed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ccefc034-b06a-451b-ba73-db9898ee8aed" (UID: "ccefc034-b06a-451b-ba73-db9898ee8aed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:31:39.823644 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.823620    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccefc034-b06a-451b-ba73-db9898ee8aed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ccefc034-b06a-451b-ba73-db9898ee8aed" (UID: "ccefc034-b06a-451b-ba73-db9898ee8aed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:31:39.823759 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.823699    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccefc034-b06a-451b-ba73-db9898ee8aed-kube-api-access-zqgdm" (OuterVolumeSpecName: "kube-api-access-zqgdm") pod "ccefc034-b06a-451b-ba73-db9898ee8aed" (UID: "ccefc034-b06a-451b-ba73-db9898ee8aed"). InnerVolumeSpecName "kube-api-access-zqgdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:31:39.823759 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.823729    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccefc034-b06a-451b-ba73-db9898ee8aed-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ccefc034-b06a-451b-ba73-db9898ee8aed" (UID: "ccefc034-b06a-451b-ba73-db9898ee8aed"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:31:39.848050 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.848025    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl"]
Sep 08 15:31:39.852369 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.852348    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-49bfbc48-gqvjl"]
Sep 08 15:31:39.923243 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.923211    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccefc034-b06a-451b-ba73-db9898ee8aed-metrics-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:39.923243 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.923238    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccefc034-b06a-451b-ba73-db9898ee8aed-openshift-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:39.923243 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.923249    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqgdm\" (UniqueName: \"kubernetes.io/projected/ccefc034-b06a-451b-ba73-db9898ee8aed-kube-api-access-zqgdm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:39.923465 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:39.923259    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccefc034-b06a-451b-ba73-db9898ee8aed-maas-api-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:40.304797 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:40.304764    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:40.432096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:40.432067    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3795a140-a911-40ae-a195-b93fb48a3fd4" path="/var/lib/kubelet/pods/3795a140-a911-40ae-a195-b93fb48a3fd4/volumes"
Sep 08 15:31:40.494269 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:40.494240    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:40.536819 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:40.536788    2575 generic.go:356] "Generic (PLEG): container finished" podID="ccefc034-b06a-451b-ba73-db9898ee8aed" containerID="6a70b0f1fd525c14a96ee84a66bb4e5de7e775e26f8e3ad360e0e9db3a073a9a" exitCode=0
Sep 08 15:31:40.536933 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:40.536829    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"
Sep 08 15:31:40.536933 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:40.536865    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq" event={"ID":"ccefc034-b06a-451b-ba73-db9898ee8aed","Type":"ContainerDied","Data":"6a70b0f1fd525c14a96ee84a66bb4e5de7e775e26f8e3ad360e0e9db3a073a9a"}
Sep 08 15:31:40.536933 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:40.536918    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq" event={"ID":"ccefc034-b06a-451b-ba73-db9898ee8aed","Type":"ContainerDied","Data":"996738924d7019d5449f9f518a45cf989ea880f80462d959b33806e2e4db9f09"}
Sep 08 15:31:40.537041 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:40.536947    2575 scope.go:117] "RemoveContainer" containerID="6a70b0f1fd525c14a96ee84a66bb4e5de7e775e26f8e3ad360e0e9db3a073a9a"
Sep 08 15:31:40.546100 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:40.546083    2575 scope.go:117] "RemoveContainer" containerID="6a70b0f1fd525c14a96ee84a66bb4e5de7e775e26f8e3ad360e0e9db3a073a9a"
Sep 08 15:31:40.546377 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:31:40.546357    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a70b0f1fd525c14a96ee84a66bb4e5de7e775e26f8e3ad360e0e9db3a073a9a\": container with ID starting with 6a70b0f1fd525c14a96ee84a66bb4e5de7e775e26f8e3ad360e0e9db3a073a9a not found: ID does not exist" containerID="6a70b0f1fd525c14a96ee84a66bb4e5de7e775e26f8e3ad360e0e9db3a073a9a"
Sep 08 15:31:40.546450 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:40.546385    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a70b0f1fd525c14a96ee84a66bb4e5de7e775e26f8e3ad360e0e9db3a073a9a"} err="failed to get container status \"6a70b0f1fd525c14a96ee84a66bb4e5de7e775e26f8e3ad360e0e9db3a073a9a\": rpc error: code = NotFound desc = could not find container \"6a70b0f1fd525c14a96ee84a66bb4e5de7e775e26f8e3ad360e0e9db3a073a9a\": container with ID starting with 6a70b0f1fd525c14a96ee84a66bb4e5de7e775e26f8e3ad360e0e9db3a073a9a not found: ID does not exist"
Sep 08 15:31:40.590409 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:40.590345    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"]
Sep 08 15:31:40.598377 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:40.598352    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-49bfbc48-5ccb768fd6-nmxvq"]
Sep 08 15:31:42.431584 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:42.431552    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccefc034-b06a-451b-ba73-db9898ee8aed" path="/var/lib/kubelet/pods/ccefc034-b06a-451b-ba73-db9898ee8aed/volumes"
Sep 08 15:31:46.541104 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:46.541069    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:47.867442 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:47.867407    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:48.129845 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:48.129761    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:48.137365 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:48.137340    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"]
Sep 08 15:31:48.137571 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:48.137553    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy" containerID="cri-o://3a2d8b1fd04cbce71d1c9b0ba90b982fe9b5a3345c0807c0df67e6d924df105a" gracePeriod=30
Sep 08 15:31:48.418774 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:48.418683    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:48.769363 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:48.769277    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:31:48.960641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:48.960613    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:49.211622 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:49.211588    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:50.966775 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:50.966737    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:52.856159 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:52.856119    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:53.391945 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.391918    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:31:53.454317 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454220    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-credential-socket\") pod \"aec05402-1b16-4942-b4e4-7d28c306c0b8\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") "
Sep 08 15:31:53.454317 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454280    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-podinfo\") pod \"aec05402-1b16-4942-b4e4-7d28c306c0b8\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") "
Sep 08 15:31:53.454562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454324    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-workload-certs\") pod \"aec05402-1b16-4942-b4e4-7d28c306c0b8\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") "
Sep 08 15:31:53.454562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454373    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-workload-socket\") pod \"aec05402-1b16-4942-b4e4-7d28c306c0b8\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") "
Sep 08 15:31:53.454562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454419    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-token\") pod \"aec05402-1b16-4942-b4e4-7d28c306c0b8\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") "
Sep 08 15:31:53.454562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454455    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aec05402-1b16-4942-b4e4-7d28c306c0b8-istiod-ca-cert\") pod \"aec05402-1b16-4942-b4e4-7d28c306c0b8\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") "
Sep 08 15:31:53.454562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454503    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-data\") pod \"aec05402-1b16-4942-b4e4-7d28c306c0b8\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") "
Sep 08 15:31:53.454562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454538    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aec05402-1b16-4942-b4e4-7d28c306c0b8" (UID: "aec05402-1b16-4942-b4e4-7d28c306c0b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:31:53.454562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454559    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqb7m\" (UniqueName: \"kubernetes.io/projected/aec05402-1b16-4942-b4e4-7d28c306c0b8-kube-api-access-wqb7m\") pod \"aec05402-1b16-4942-b4e4-7d28c306c0b8\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") "
Sep 08 15:31:53.454909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454644    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-envoy\") pod \"aec05402-1b16-4942-b4e4-7d28c306c0b8\" (UID: \"aec05402-1b16-4942-b4e4-7d28c306c0b8\") "
Sep 08 15:31:53.454909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454656    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aec05402-1b16-4942-b4e4-7d28c306c0b8" (UID: "aec05402-1b16-4942-b4e4-7d28c306c0b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:31:53.454909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454695    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aec05402-1b16-4942-b4e4-7d28c306c0b8" (UID: "aec05402-1b16-4942-b4e4-7d28c306c0b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:31:53.454909 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454900    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "aec05402-1b16-4942-b4e4-7d28c306c0b8" (UID: "aec05402-1b16-4942-b4e4-7d28c306c0b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:31:53.455124 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.454997    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aec05402-1b16-4942-b4e4-7d28c306c0b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aec05402-1b16-4942-b4e4-7d28c306c0b8" (UID: "aec05402-1b16-4942-b4e4-7d28c306c0b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:31:53.455124 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.455045    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:53.455124 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.455067    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:53.455124 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.455082    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:53.455124 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.455095    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:53.456595 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.456566    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aec05402-1b16-4942-b4e4-7d28c306c0b8" (UID: "aec05402-1b16-4942-b4e4-7d28c306c0b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:31:53.456702 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.456685    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aec05402-1b16-4942-b4e4-7d28c306c0b8" (UID: "aec05402-1b16-4942-b4e4-7d28c306c0b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:31:53.457096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.457073    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "aec05402-1b16-4942-b4e4-7d28c306c0b8" (UID: "aec05402-1b16-4942-b4e4-7d28c306c0b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:31:53.457201 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.457103    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aec05402-1b16-4942-b4e4-7d28c306c0b8-kube-api-access-wqb7m" (OuterVolumeSpecName: "kube-api-access-wqb7m") pod "aec05402-1b16-4942-b4e4-7d28c306c0b8" (UID: "aec05402-1b16-4942-b4e4-7d28c306c0b8"). InnerVolumeSpecName "kube-api-access-wqb7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:31:53.555603 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.555567    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:53.555603 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.555595    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:53.555603 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.555604    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aec05402-1b16-4942-b4e4-7d28c306c0b8-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:53.555603 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.555613    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqb7m\" (UniqueName: \"kubernetes.io/projected/aec05402-1b16-4942-b4e4-7d28c306c0b8-kube-api-access-wqb7m\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:53.555868 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.555622    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aec05402-1b16-4942-b4e4-7d28c306c0b8-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:31:53.614450 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.614415    2575 generic.go:356] "Generic (PLEG): container finished" podID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerID="3a2d8b1fd04cbce71d1c9b0ba90b982fe9b5a3345c0807c0df67e6d924df105a" exitCode=0
Sep 08 15:31:53.614638 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.614466    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"
Sep 08 15:31:53.614638 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.614479    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" event={"ID":"aec05402-1b16-4942-b4e4-7d28c306c0b8","Type":"ContainerDied","Data":"3a2d8b1fd04cbce71d1c9b0ba90b982fe9b5a3345c0807c0df67e6d924df105a"}
Sep 08 15:31:53.614638 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.614547    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb" event={"ID":"aec05402-1b16-4942-b4e4-7d28c306c0b8","Type":"ContainerDied","Data":"915f35d37f80877b5cb9a6e2fcbc1162cd6c508dc4b037b24608afcbfe1729f9"}
Sep 08 15:31:53.614638 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.614578    2575 scope.go:117] "RemoveContainer" containerID="3a2d8b1fd04cbce71d1c9b0ba90b982fe9b5a3345c0807c0df67e6d924df105a"
Sep 08 15:31:53.623919 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.623901    2575 scope.go:117] "RemoveContainer" containerID="2e7cf41757e7fa112e6a3d406297feea62a4d63d62c9846f0f4f5b6a0ff0a1bd"
Sep 08 15:31:53.632080 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.632059    2575 scope.go:117] "RemoveContainer" containerID="3a2d8b1fd04cbce71d1c9b0ba90b982fe9b5a3345c0807c0df67e6d924df105a"
Sep 08 15:31:53.632399 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:31:53.632380    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a2d8b1fd04cbce71d1c9b0ba90b982fe9b5a3345c0807c0df67e6d924df105a\": container with ID starting with 3a2d8b1fd04cbce71d1c9b0ba90b982fe9b5a3345c0807c0df67e6d924df105a not found: ID does not exist" containerID="3a2d8b1fd04cbce71d1c9b0ba90b982fe9b5a3345c0807c0df67e6d924df105a"
Sep 08 15:31:53.632450 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.632409    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a2d8b1fd04cbce71d1c9b0ba90b982fe9b5a3345c0807c0df67e6d924df105a"} err="failed to get container status \"3a2d8b1fd04cbce71d1c9b0ba90b982fe9b5a3345c0807c0df67e6d924df105a\": rpc error: code = NotFound desc = could not find container \"3a2d8b1fd04cbce71d1c9b0ba90b982fe9b5a3345c0807c0df67e6d924df105a\": container with ID starting with 3a2d8b1fd04cbce71d1c9b0ba90b982fe9b5a3345c0807c0df67e6d924df105a not found: ID does not exist"
Sep 08 15:31:53.632450 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.632426    2575 scope.go:117] "RemoveContainer" containerID="2e7cf41757e7fa112e6a3d406297feea62a4d63d62c9846f0f4f5b6a0ff0a1bd"
Sep 08 15:31:53.632685 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:31:53.632667    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e7cf41757e7fa112e6a3d406297feea62a4d63d62c9846f0f4f5b6a0ff0a1bd\": container with ID starting with 2e7cf41757e7fa112e6a3d406297feea62a4d63d62c9846f0f4f5b6a0ff0a1bd not found: ID does not exist" containerID="2e7cf41757e7fa112e6a3d406297feea62a4d63d62c9846f0f4f5b6a0ff0a1bd"
Sep 08 15:31:53.632752 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.632694    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e7cf41757e7fa112e6a3d406297feea62a4d63d62c9846f0f4f5b6a0ff0a1bd"} err="failed to get container status \"2e7cf41757e7fa112e6a3d406297feea62a4d63d62c9846f0f4f5b6a0ff0a1bd\": rpc error: code = NotFound desc = could not find container \"2e7cf41757e7fa112e6a3d406297feea62a4d63d62c9846f0f4f5b6a0ff0a1bd\": container with ID starting with 2e7cf41757e7fa112e6a3d406297feea62a4d63d62c9846f0f4f5b6a0ff0a1bd not found: ID does not exist"
Sep 08 15:31:53.646210 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.646182    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"]
Sep 08 15:31:53.662930 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:53.662901    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-49bfbc48-openshift-default-5dcfdfbd76-25wgb"]
Sep 08 15:31:54.431421 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.431387    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" path="/var/lib/kubelet/pods/aec05402-1b16-4942-b4e4-7d28c306c0b8/volumes"
Sep 08 15:31:54.693881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.693803    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"]
Sep 08 15:31:54.694190 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.694178    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccefc034-b06a-451b-ba73-db9898ee8aed" containerName="maas-api"
Sep 08 15:31:54.694232 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.694191    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccefc034-b06a-451b-ba73-db9898ee8aed" containerName="maas-api"
Sep 08 15:31:54.694232 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.694209    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy"
Sep 08 15:31:54.694232 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.694214    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy"
Sep 08 15:31:54.694232 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.694225    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy"
Sep 08 15:31:54.694232 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.694230    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy"
Sep 08 15:31:54.694388 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.694240    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3795a140-a911-40ae-a195-b93fb48a3fd4" containerName="revoke-keys"
Sep 08 15:31:54.694388 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.694245    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3795a140-a911-40ae-a195-b93fb48a3fd4" containerName="revoke-keys"
Sep 08 15:31:54.694388 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.694301    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy"
Sep 08 15:31:54.694388 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.694312    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3795a140-a911-40ae-a195-b93fb48a3fd4" containerName="revoke-keys"
Sep 08 15:31:54.694388 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.694318    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccefc034-b06a-451b-ba73-db9898ee8aed" containerName="maas-api"
Sep 08 15:31:54.697450 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.697431    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.701452 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.701431    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2e082e\"/\"auth-test-model-2e082e-kserve-self-signed-certs\""
Sep 08 15:31:54.702330 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.702314    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2e082e\"/\"default-dockercfg-bq58p\""
Sep 08 15:31:54.702587 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.702566    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2e082e\"/\"openshift-service-ca.crt\""
Sep 08 15:31:54.702813 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.702795    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2e082e\"/\"kube-root-ca.crt\""
Sep 08 15:31:54.734404 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.734373    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"]
Sep 08 15:31:54.766446 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.766412    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44zcg\" (UniqueName: \"kubernetes.io/projected/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-kube-api-access-44zcg\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.766630 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.766530    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-tmp-dir\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.766715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.766627    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-home\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.766715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.766683    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-model-cache\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.766715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.766707    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-tls-certs\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.766898 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.766761    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-dshm\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.867425 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.867390    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44zcg\" (UniqueName: \"kubernetes.io/projected/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-kube-api-access-44zcg\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.867653 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.867446    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-tmp-dir\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.867653 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.867520    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-home\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.867653 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.867557    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-model-cache\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.867653 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.867582    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-tls-certs\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.867653 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.867635    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-dshm\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.867913 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.867893    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-home\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.867963 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.867942    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-tmp-dir\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.868002 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.867982    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-model-cache\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.869863 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.869842    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-dshm\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.870056 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.870038    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-tls-certs\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.880085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.880065    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44zcg\" (UniqueName: \"kubernetes.io/projected/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-kube-api-access-44zcg\") pod \"auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:54.892934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:54.892908    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:55.006393 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:55.006299    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:31:55.290612 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:55.290585    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"]
Sep 08 15:31:55.291451 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:31:55.291421    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e3fb9f1_cfc4_48c6_b67f_f206a17e1cfd.slice/crio-e32be9fd902b759880f1d5461a7338fdcda7fafe692253b5e5990ff8ce724457 WatchSource:0}: Error finding container e32be9fd902b759880f1d5461a7338fdcda7fafe692253b5e5990ff8ce724457: Status 404 returned error can't find the container with id e32be9fd902b759880f1d5461a7338fdcda7fafe692253b5e5990ff8ce724457
Sep 08 15:31:55.628789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:55.628756    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk" event={"ID":"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd","Type":"ContainerStarted","Data":"e12015454b7cb9868882ab67f2d19c06470b05ab259a99a78aa76c1c945fb9ec"}
Sep 08 15:31:55.629228 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:55.628807    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk" event={"ID":"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd","Type":"ContainerStarted","Data":"e32be9fd902b759880f1d5461a7338fdcda7fafe692253b5e5990ff8ce724457"}
Sep 08 15:31:55.660145 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:55.660098    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk" podStartSLOduration=1.6600818899999998 podStartE2EDuration="1.66008189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:31:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:31:55.659681638 +0000 UTC m=+1363.745631683" watchObservedRunningTime="2026-09-08 15:31:55.66008189 +0000 UTC m=+1363.746031932"
Sep 08 15:31:56.526369 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:56.526339    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:56.670131 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:56.670093    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:56.782562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:56.782459    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:57.019034 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:57.018988    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:31:57.790669 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:31:57.790634    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:32:04.022971 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:04.022921    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:04.262759 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:04.262726    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:05.006947 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.006905    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:32:05.007144 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.006961    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:32:05.015198 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.015159    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:05.019827 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.019802    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:32:05.449654 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.449617    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"]
Sep 08 15:32:05.450092 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.450078    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="aec05402-1b16-4942-b4e4-7d28c306c0b8" containerName="istio-proxy"
Sep 08 15:32:05.452869 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.452840    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.475728 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.475700    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-149fdb-openshift-default-dockercfg-lvswq\""
Sep 08 15:32:05.500970 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.500932    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"]
Sep 08 15:32:05.576294 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.576252    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istiod-ca-cert\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.576294 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.576295    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-workload-socket\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.576547 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.576362    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-credential-socket\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.576547 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.576387    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-envoy\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.576547 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.576421    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-workload-certs\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.576547 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.576447    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-token\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.576547 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.576481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-data\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.576547 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.576535    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhr2m\" (UniqueName: \"kubernetes.io/projected/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-kube-api-access-fhr2m\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.576769 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.576586    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-podinfo\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.579337 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.579309    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:05.677755 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.677714    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-data\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.677936 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.677790    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhr2m\" (UniqueName: \"kubernetes.io/projected/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-kube-api-access-fhr2m\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.677936 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.677869    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-podinfo\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.678027 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.677933    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istiod-ca-cert\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.678027 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.677964    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-workload-socket\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.678027 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.678010    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-credential-socket\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.678183 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.678038    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-envoy\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.678183 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.678086    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-workload-certs\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.678183 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.678123    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-token\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.678358 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.678295    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-workload-socket\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.678418 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.678386    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-credential-socket\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.678418 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.678406    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-data\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.678717 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.678692    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-workload-certs\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.678862 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.678808    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istiod-ca-cert\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.680702 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.680642    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-envoy\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.680852 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.680838    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-podinfo\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.692643 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.692605    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhr2m\" (UniqueName: \"kubernetes.io/projected/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-kube-api-access-fhr2m\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.697751 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.697723    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:32:05.700181 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.700115    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-token\") pod \"e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") " pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.763798 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.763762    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:05.934374 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:05.934307    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:06.076961 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:06.076931    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"]
Sep 08 15:32:06.078735 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:32:06.078707    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ff18c8_3caf_4048_b7d8_3c5699caadcc.slice/crio-b1ec0fd636118129058803b8ee340cf35760ae874381ebdc2357e4a21afc6ee2 WatchSource:0}: Error finding container b1ec0fd636118129058803b8ee340cf35760ae874381ebdc2357e4a21afc6ee2: Status 404 returned error can't find the container with id b1ec0fd636118129058803b8ee340cf35760ae874381ebdc2357e4a21afc6ee2
Sep 08 15:32:06.080952 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:06.080918    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:32:06.081059 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:06.080995    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:32:06.081059 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:06.081038    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:32:06.692451 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:06.692418    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" event={"ID":"d7ff18c8-3caf-4048-b7d8-3c5699caadcc","Type":"ContainerStarted","Data":"e8170b190ec41340512f9477d78b52f884de6e8ed7d1abfd112b20fd4e760c39"}
Sep 08 15:32:06.692881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:06.692476    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" event={"ID":"d7ff18c8-3caf-4048-b7d8-3c5699caadcc","Type":"ContainerStarted","Data":"b1ec0fd636118129058803b8ee340cf35760ae874381ebdc2357e4a21afc6ee2"}
Sep 08 15:32:06.727621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:06.727566    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" podStartSLOduration=1.7275467469999999 podStartE2EDuration="1.727546747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:32:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:06.726729568 +0000 UTC m=+1374.812679613" watchObservedRunningTime="2026-09-08 15:32:06.727546747 +0000 UTC m=+1374.813496782"
Sep 08 15:32:06.764827 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:06.764790    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:07.770022 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:07.769988    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:08.704902 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:08.704869    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:08.705952 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:08.705930    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:08.824168 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:08.824133    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:08.911627 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:08.911592    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:10.041991 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:10.041956    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:10.147421 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:10.147381    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:10.400621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:10.400590    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:10.710377 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:10.710263    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:10.807996 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:10.807958    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:12.839091 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:12.839056    2575 scope.go:117] "RemoveContainer" containerID="60b932e9c6c9fd066bb75547077be955d7a751003d705f12425e2760077d1556"
Sep 08 15:32:15.751670 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:15.751637    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:32:19.451042 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.451000    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"]
Sep 08 15:32:19.456542 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.456518    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.463171 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.463134    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-bc9cdf\"/\"default-dockercfg-xpclp\""
Sep 08 15:32:19.463357 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.463274    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-bc9cdf\"/\"kube-root-ca.crt\""
Sep 08 15:32:19.463357 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.463281    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-bc9cdf\"/\"openshift-service-ca.crt\""
Sep 08 15:32:19.463571 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.463387    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-bc9cdf\"/\"auth-test-model-bc9cdf-kserve-self-signed-certs\""
Sep 08 15:32:19.468033 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.468008    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"]
Sep 08 15:32:19.508518 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.502537    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:19.639348 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.639311    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-tls-certs\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.639556 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.639355    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdnxx\" (UniqueName: \"kubernetes.io/projected/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-kube-api-access-qdnxx\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.639556 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.639482    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-tmp-dir\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.639721 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.639583    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-model-cache\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.639721 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.639630    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-dshm\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.639721 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.639656    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-home\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.740778 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.740684    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-tmp-dir\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.740778 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.740733    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-model-cache\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.740778 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.740766    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-dshm\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.741068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.740790    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-home\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.741068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.740844    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-tls-certs\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.741068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.740882    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdnxx\" (UniqueName: \"kubernetes.io/projected/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-kube-api-access-qdnxx\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.741264 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.741177    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-tmp-dir\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.741264 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.741220    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-model-cache\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.741360 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.741259    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-home\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.743199 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.743181    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-dshm\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.743372 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.743357    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-tls-certs\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.751906 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.751883    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdnxx\" (UniqueName: \"kubernetes.io/projected/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-kube-api-access-qdnxx\") pod \"auth-test-model-bc9cdf-kserve-fd77b45c9-687tc\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:19.770886 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:19.770861    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:20.073016 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:20.072987    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"]
Sep 08 15:32:20.075783 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:32:20.075748    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6895f35_f7d0_4dcb_aa11_c7c5e694afa9.slice/crio-e002c4259399cc373e7d0b5509e855e797f9ae68d07d6e4789c3ea39d7eb3499 WatchSource:0}: Error finding container e002c4259399cc373e7d0b5509e855e797f9ae68d07d6e4789c3ea39d7eb3499: Status 404 returned error can't find the container with id e002c4259399cc373e7d0b5509e855e797f9ae68d07d6e4789c3ea39d7eb3499
Sep 08 15:32:20.783385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:20.783344    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc" event={"ID":"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9","Type":"ContainerStarted","Data":"9001bac29292bfa2c853d8d0d413d673e0ae3baef7280541359fcfa6155c37cf"}
Sep 08 15:32:20.783385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:20.783391    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc" event={"ID":"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9","Type":"ContainerStarted","Data":"e002c4259399cc373e7d0b5509e855e797f9ae68d07d6e4789c3ea39d7eb3499"}
Sep 08 15:32:20.805382 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:20.805305    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc" podStartSLOduration=1.8052807419999999 podStartE2EDuration="1.805280742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:32:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:20.801376368 +0000 UTC m=+1388.887326411" watchObservedRunningTime="2026-09-08 15:32:20.805280742 +0000 UTC m=+1388.891230784"
Sep 08 15:32:24.746689 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:24.746637    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 15:32:24.747068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:24.746709    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 15:32:24.955602 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:24.955564    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:32:25.812912 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:25.812873    2575 generic.go:356] "Generic (PLEG): container finished" podID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerID="e8170b190ec41340512f9477d78b52f884de6e8ed7d1abfd112b20fd4e760c39" exitCode=0
Sep 08 15:32:25.813339 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:25.812947    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" event={"ID":"d7ff18c8-3caf-4048-b7d8-3c5699caadcc","Type":"ContainerDied","Data":"e8170b190ec41340512f9477d78b52f884de6e8ed7d1abfd112b20fd4e760c39"}
Sep 08 15:32:25.813381 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:25.813348    2575 scope.go:117] "RemoveContainer" containerID="e8170b190ec41340512f9477d78b52f884de6e8ed7d1abfd112b20fd4e760c39"
Sep 08 15:32:25.814626 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:25.814587    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:32:25.814746 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:25.814677    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:32:25.814746 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:25.814731    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:32:26.822304 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:26.822265    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" event={"ID":"d7ff18c8-3caf-4048-b7d8-3c5699caadcc","Type":"ContainerStarted","Data":"d8f4f4531382642356626409f3b332411f61c0ef02a721bfa45775091ef9c723"}
Sep 08 15:32:27.764962 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:27.764910    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:27.765170 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:27.765145    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 08 15:32:27.765220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:27.765197    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 08 15:32:28.764707 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:28.764668    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 08 15:32:28.765133 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:28.764729    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 08 15:32:29.764863 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:29.764827    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 08 15:32:29.765266 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:29.764884    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 08 15:32:29.771016 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:29.770982    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:29.771187 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:29.771037    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:29.783577 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:29.783546    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:29.851542 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:29.851511    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:32:30.764188 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:30.764141    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 08 15:32:30.764409 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:30.764209    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 08 15:32:31.764920 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:31.764876    2575 patch_prober.go:28] interesting pod/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 08 15:32:31.765325 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:31.764940    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 08 15:32:32.814723 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:32.814687    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:32.815261 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:32.815017    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:32.815904 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:32.815879    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:32:33.598219 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:33.598167    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:33.857757 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:33.857715    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:34.230090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:34.229979    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:34.567532 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:34.567410    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:35.787697 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:35.787646    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:38.123850 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:38.123811    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:38.228486 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:38.228448    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:39.245220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.245180    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:32:39.248508 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.248463    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"]
Sep 08 15:32:39.251326 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.251309    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:39.264037 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.264012    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-04285b-serving-cert\""
Sep 08 15:32:39.270987 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.270953    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"]
Sep 08 15:32:39.326611 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.326573    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:39.353789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.353743    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:39.353988 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.353797    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:39.353988 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.353928    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50535ed2-f951-4199-8762-278e26fa046e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:39.354107 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.354053    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29bnz\" (UniqueName: \"kubernetes.io/projected/50535ed2-f951-4199-8762-278e26fa046e-kube-api-access-29bnz\") pod \"maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:39.455196 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.455143    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29bnz\" (UniqueName: \"kubernetes.io/projected/50535ed2-f951-4199-8762-278e26fa046e-kube-api-access-29bnz\") pod \"maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:39.455410 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.455247    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:39.455410 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.455278    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:39.455410 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.455325    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50535ed2-f951-4199-8762-278e26fa046e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:39.455410 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:32:39.455391    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-serving-cert: secret "maas-api-e2e-shared-b-w2-04285b-serving-cert" not found
Sep 08 15:32:39.455677 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:32:39.455457    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-maas-api-tls podName:50535ed2-f951-4199-8762-278e26fa046e nodeName:}" failed. No retries permitted until 2026-09-08 15:32:39.955440241 +0000 UTC m=+1408.041390261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-maas-api-tls") pod "maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc" (UID: "50535ed2-f951-4199-8762-278e26fa046e") : secret "maas-api-e2e-shared-b-w2-04285b-serving-cert" not found
Sep 08 15:32:39.456025 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.456007    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50535ed2-f951-4199-8762-278e26fa046e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:39.457812 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.457788    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:39.471369 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.471341    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29bnz\" (UniqueName: \"kubernetes.io/projected/50535ed2-f951-4199-8762-278e26fa046e-kube-api-access-29bnz\") pod \"maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:39.475993 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.475964    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:39.618822 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.618784    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:39.961966 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.961870    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:39.964314 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:39.964283    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:40.109452 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:40.109415    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:40.163809 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:40.163774    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:40.446093 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:40.446062    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"]
Sep 08 15:32:40.447825 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:32:40.447803    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50535ed2_f951_4199_8762_278e26fa046e.slice/crio-5cac3addadf742bf7c78ed7f2fa8bbb72ae675bb132482391e16642f8d4b94b1 WatchSource:0}: Error finding container 5cac3addadf742bf7c78ed7f2fa8bbb72ae675bb132482391e16642f8d4b94b1: Status 404 returned error can't find the container with id 5cac3addadf742bf7c78ed7f2fa8bbb72ae675bb132482391e16642f8d4b94b1
Sep 08 15:32:40.904108 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:40.904068    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc" event={"ID":"50535ed2-f951-4199-8762-278e26fa046e","Type":"ContainerStarted","Data":"5cac3addadf742bf7c78ed7f2fa8bbb72ae675bb132482391e16642f8d4b94b1"}
Sep 08 15:32:40.978770 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:40.978729    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:32:42.917625 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:42.917586    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc" event={"ID":"50535ed2-f951-4199-8762-278e26fa046e","Type":"ContainerStarted","Data":"f197645b36c859850a9041178c783c53e5062f96923875aa7745d44daf8f8c6d"}
Sep 08 15:32:42.918038 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:42.917745    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:42.940686 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:42.940626    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc" podStartSLOduration=2.5340562970000002 podStartE2EDuration="3.940606419s" totalImagesPullingTime="1.406550122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:42.938644279 +0000 UTC m=+1411.024594322" watchObservedRunningTime="2026-09-08 15:32:42.940606419 +0000 UTC m=+1411.026556460"
Sep 08 15:32:48.931264 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:48.931231    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:32:52.949220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:32:52.949176    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:33:03.264389 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:03.264349    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:04.642605 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:04.642569    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:33:12.571297 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:12.571260    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:12.869476 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:12.869441    2575 scope.go:117] "RemoveContainer" containerID="df5951839345fda087a2730496055e8cd569a4520a302b6fa95181ec3a48a5ea"
Sep 08 15:33:17.640449 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.640407    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:17.952402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.952314    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"]
Sep 08 15:33:17.955523 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.955482    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:17.958228 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.958200    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cff875c0-c818-41c2-86a6-11f297288a35-istio-token\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:17.958360 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.958263    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-credential-socket\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:17.958360 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.958336    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-istio-data\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:17.958438 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.958399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cff875c0-c818-41c2-86a6-11f297288a35-istio-podinfo\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:17.958516 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.958464    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqnbr\" (UniqueName: \"kubernetes.io/projected/cff875c0-c818-41c2-86a6-11f297288a35-kube-api-access-tqnbr\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:17.958572 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.958532    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-workload-socket\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:17.958623 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.958566    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-istio-envoy\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:17.958623 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.958614    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-workload-certs\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:17.958708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.958663    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cff875c0-c818-41c2-86a6-11f297288a35-istiod-ca-cert\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:17.960033 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.960016    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ae34c54a-openshift-default-dockercfg-hldrk\""
Sep 08 15:33:17.971405 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.971376    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"]
Sep 08 15:33:17.981913 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:17.981871    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:18.059777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.059731    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cff875c0-c818-41c2-86a6-11f297288a35-istio-token\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.060426 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.060395    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-credential-socket\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.060689 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.060669    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-istio-data\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.061380 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.061156    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-credential-socket\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.061380 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.061237    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-istio-data\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.061769 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.061316    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cff875c0-c818-41c2-86a6-11f297288a35-istio-podinfo\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.061937 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.061877    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqnbr\" (UniqueName: \"kubernetes.io/projected/cff875c0-c818-41c2-86a6-11f297288a35-kube-api-access-tqnbr\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.062190 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.062151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-workload-socket\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.062295 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.062241    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-istio-envoy\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.062295 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.062285    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-workload-certs\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.062414 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.062338    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cff875c0-c818-41c2-86a6-11f297288a35-istiod-ca-cert\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.062467 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.062455    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-workload-socket\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.063022 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.062999    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-workload-certs\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.063806 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.063784    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cff875c0-c818-41c2-86a6-11f297288a35-istiod-ca-cert\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.064934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.064914    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cff875c0-c818-41c2-86a6-11f297288a35-istio-podinfo\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.065094 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.064999    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-istio-envoy\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.068740 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.068715    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cff875c0-c818-41c2-86a6-11f297288a35-istio-token\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.070060 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.070042    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqnbr\" (UniqueName: \"kubernetes.io/projected/cff875c0-c818-41c2-86a6-11f297288a35-kube-api-access-tqnbr\") pod \"e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") " pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.151811 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.151772    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:18.266661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.266564    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:18.546736 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.546709    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"]
Sep 08 15:33:18.549172 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:33:18.549145    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcff875c0_c818_41c2_86a6_11f297288a35.slice/crio-68dff083e2c0b4af152eddeae9f89730a0d94ea53d151bec657f382b7979073e WatchSource:0}: Error finding container 68dff083e2c0b4af152eddeae9f89730a0d94ea53d151bec657f382b7979073e: Status 404 returned error can't find the container with id 68dff083e2c0b4af152eddeae9f89730a0d94ea53d151bec657f382b7979073e
Sep 08 15:33:18.551634 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.551594    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:33:18.551733 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.551694    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:33:18.551797 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:18.551743    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:33:19.121165 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:19.121127    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp" event={"ID":"cff875c0-c818-41c2-86a6-11f297288a35","Type":"ContainerStarted","Data":"d7e637326aa7b35c8baaeef59fbe7576a0ffc3982c9ff1d6fcbdce015d4044c1"}
Sep 08 15:33:19.121599 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:19.121184    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp" event={"ID":"cff875c0-c818-41c2-86a6-11f297288a35","Type":"ContainerStarted","Data":"68dff083e2c0b4af152eddeae9f89730a0d94ea53d151bec657f382b7979073e"}
Sep 08 15:33:19.164177 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:19.164117    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp" podStartSLOduration=2.164100259 podStartE2EDuration="2.164100259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:33:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:33:19.16320532 +0000 UTC m=+1447.249155361" watchObservedRunningTime="2026-09-08 15:33:19.164100259 +0000 UTC m=+1447.250050300"
Sep 08 15:33:19.267230 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:19.267188    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:19.273565 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:19.273484    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:20.127071 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:20.127039    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:20.128211 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:20.128192    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:21.170010 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:21.169967    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:21.259642 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:21.259606    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:22.306279 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.306243    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"]
Sep 08 15:33:22.309099 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.309076    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.312130 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.312108    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ae34c54a-serving-cert\""
Sep 08 15:33:22.320756 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.320724    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"]
Sep 08 15:33:22.371023 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.370976    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:22.410773 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.410738    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc73d9eb-d418-440a-95ed-2aa2706172f0-maas-api-tls\") pod \"maas-api-e2e-mt-ae34c54a-c5865764-9lhgx\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.410953 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.410793    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x9hp\" (UniqueName: \"kubernetes.io/projected/bc73d9eb-d418-440a-95ed-2aa2706172f0-kube-api-access-2x9hp\") pod \"maas-api-e2e-mt-ae34c54a-c5865764-9lhgx\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.410953 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.410879    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc73d9eb-d418-440a-95ed-2aa2706172f0-openshift-service-ca\") pod \"maas-api-e2e-mt-ae34c54a-c5865764-9lhgx\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.410953 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.410945    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc73d9eb-d418-440a-95ed-2aa2706172f0-metrics-certs\") pod \"maas-api-e2e-mt-ae34c54a-c5865764-9lhgx\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.511758 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.511721    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x9hp\" (UniqueName: \"kubernetes.io/projected/bc73d9eb-d418-440a-95ed-2aa2706172f0-kube-api-access-2x9hp\") pod \"maas-api-e2e-mt-ae34c54a-c5865764-9lhgx\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.511951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.511767    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc73d9eb-d418-440a-95ed-2aa2706172f0-openshift-service-ca\") pod \"maas-api-e2e-mt-ae34c54a-c5865764-9lhgx\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.511951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.511825    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc73d9eb-d418-440a-95ed-2aa2706172f0-metrics-certs\") pod \"maas-api-e2e-mt-ae34c54a-c5865764-9lhgx\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.512058 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.511988    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc73d9eb-d418-440a-95ed-2aa2706172f0-maas-api-tls\") pod \"maas-api-e2e-mt-ae34c54a-c5865764-9lhgx\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.512594 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.512570    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc73d9eb-d418-440a-95ed-2aa2706172f0-openshift-service-ca\") pod \"maas-api-e2e-mt-ae34c54a-c5865764-9lhgx\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.514426 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.514392    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc73d9eb-d418-440a-95ed-2aa2706172f0-maas-api-tls\") pod \"maas-api-e2e-mt-ae34c54a-c5865764-9lhgx\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.514560 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.514506    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc73d9eb-d418-440a-95ed-2aa2706172f0-metrics-certs\") pod \"maas-api-e2e-mt-ae34c54a-c5865764-9lhgx\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.522475 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.522451    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x9hp\" (UniqueName: \"kubernetes.io/projected/bc73d9eb-d418-440a-95ed-2aa2706172f0-kube-api-access-2x9hp\") pod \"maas-api-e2e-mt-ae34c54a-c5865764-9lhgx\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.539421 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.539390    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:22.622745 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.622713    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:22.641100 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.641036    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:22.883298 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.883207    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:22.909301 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:22.909273    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"]
Sep 08 15:33:22.911378 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:33:22.911351    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc73d9eb_d418_440a_95ed_2aa2706172f0.slice/crio-3b6b30874d47e7faad57a0fd9c43221682feaab7255555ea854ea6c0ee37ed6e WatchSource:0}: Error finding container 3b6b30874d47e7faad57a0fd9c43221682feaab7255555ea854ea6c0ee37ed6e: Status 404 returned error can't find the container with id 3b6b30874d47e7faad57a0fd9c43221682feaab7255555ea854ea6c0ee37ed6e
Sep 08 15:33:23.003036 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:23.002997    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:23.095794 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:23.095758    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:33:23.148351 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:23.148243    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx" event={"ID":"bc73d9eb-d418-440a-95ed-2aa2706172f0","Type":"ContainerStarted","Data":"3b6b30874d47e7faad57a0fd9c43221682feaab7255555ea854ea6c0ee37ed6e"}
Sep 08 15:33:24.155729 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:24.155695    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx" event={"ID":"bc73d9eb-d418-440a-95ed-2aa2706172f0","Type":"ContainerStarted","Data":"da0aa64ba2ace139a11f931067fb1a744d7c1743f3ed4d1d9be6f68045400b61"}
Sep 08 15:33:24.156194 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:24.155781    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:24.182641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:24.182584    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx" podStartSLOduration=1.45326429 podStartE2EDuration="2.182566161s" totalImagesPullingTime="729.301871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:33:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:33:24.180980024 +0000 UTC m=+1452.266930078" watchObservedRunningTime="2026-09-08 15:33:24.182566161 +0000 UTC m=+1452.268516202"
Sep 08 15:33:30.167058 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:30.167025    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:33:41.168818 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:41.168769    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:33:43.266620 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:43.266518    2575 generic.go:356] "Generic (PLEG): container finished" podID="cff875c0-c818-41c2-86a6-11f297288a35" containerID="d7e637326aa7b35c8baaeef59fbe7576a0ffc3982c9ff1d6fcbdce015d4044c1" exitCode=0
Sep 08 15:33:43.266620 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:43.266592    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp" event={"ID":"cff875c0-c818-41c2-86a6-11f297288a35","Type":"ContainerDied","Data":"d7e637326aa7b35c8baaeef59fbe7576a0ffc3982c9ff1d6fcbdce015d4044c1"}
Sep 08 15:33:43.267112 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:43.267023    2575 scope.go:117] "RemoveContainer" containerID="d7e637326aa7b35c8baaeef59fbe7576a0ffc3982c9ff1d6fcbdce015d4044c1"
Sep 08 15:33:43.268275 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:43.268245    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:33:43.268366 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:43.268314    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:33:43.268366 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:43.268346    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:33:44.274440 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:44.274405    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp" event={"ID":"cff875c0-c818-41c2-86a6-11f297288a35","Type":"ContainerStarted","Data":"51fee8dfc195e41de149ce80b861015fa873f70855e592bb3f9c67a86bd00280"}
Sep 08 15:33:45.267034 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:45.267001    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:45.267315 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:45.267279    2575 patch_prober.go:28] interesting pod/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 08 15:33:45.267435 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:45.267341    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp" podUID="cff875c0-c818-41c2-86a6-11f297288a35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 08 15:33:46.221745 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:46.221706    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:33:46.267708 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:46.267666    2575 patch_prober.go:28] interesting pod/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 08 15:33:46.267873 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:46.267734    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp" podUID="cff875c0-c818-41c2-86a6-11f297288a35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 08 15:33:47.267445 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:47.267403    2575 patch_prober.go:28] interesting pod/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 08 15:33:47.267875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:47.267474    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp" podUID="cff875c0-c818-41c2-86a6-11f297288a35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 08 15:33:48.267113 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:48.267081    2575 patch_prober.go:28] interesting pod/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 08 15:33:48.267323 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:48.267135    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp" podUID="cff875c0-c818-41c2-86a6-11f297288a35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 08 15:33:49.267212 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:49.267171    2575 patch_prober.go:28] interesting pod/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 08 15:33:49.267671 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:49.267234    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp" podUID="cff875c0-c818-41c2-86a6-11f297288a35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 08 15:33:50.127889 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:50.127855    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:50.273234 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:50.273203    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:50.321323 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:50.321295    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:33:55.538831 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:33:55.538798    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:34:08.732059 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:08.732022    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:34:09.942191 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:09.942157    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"]
Sep 08 15:34:09.945400 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:09.945373    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:09.949633 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:09.949606    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ae34c54a\"/\"default-dockercfg-thq59\""
Sep 08 15:34:09.949633 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:09.949621    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ae34c54a\"/\"openshift-service-ca.crt\""
Sep 08 15:34:09.949846 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:09.949652    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ae34c54a\"/\"kube-root-ca.crt\""
Sep 08 15:34:09.949846 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:09.949621    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ae34c54a\"/\"e2e-lifecycle-model-ae34c54a-kserve-self-signed-certs\""
Sep 08 15:34:09.956604 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:09.956583    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"]
Sep 08 15:34:09.999565 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:09.999523    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:10.036676 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.036648    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-model-cache\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.036821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.036687    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bde229e-169e-4691-9941-b15f266e47a2-tls-certs\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.036821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.036706    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99cp2\" (UniqueName: \"kubernetes.io/projected/3bde229e-169e-4691-9941-b15f266e47a2-kube-api-access-99cp2\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.036821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.036803    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-home\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.036962 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.036831    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-dshm\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.036962 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.036944    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-tmp-dir\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.138218 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.138185    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-tmp-dir\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.138431 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.138250    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-model-cache\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.138431 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.138276    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bde229e-169e-4691-9941-b15f266e47a2-tls-certs\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.138431 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.138293    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99cp2\" (UniqueName: \"kubernetes.io/projected/3bde229e-169e-4691-9941-b15f266e47a2-kube-api-access-99cp2\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.138431 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.138334    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-home\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.138431 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.138350    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-dshm\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.138737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.138692    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-tmp-dir\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.138790 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.138734    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-model-cache\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.138831 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.138792    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-home\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.140728 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.140695    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-dshm\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.140952 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.140934    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bde229e-169e-4691-9941-b15f266e47a2-tls-certs\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.148808 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.148777    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99cp2\" (UniqueName: \"kubernetes.io/projected/3bde229e-169e-4691-9941-b15f266e47a2-kube-api-access-99cp2\") pod \"e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") " pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.256338 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.256265    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:10.533331 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:10.533303    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"]
Sep 08 15:34:10.535340 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:34:10.535303    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3bde229e_169e_4691_9941_b15f266e47a2.slice/crio-14a8d7bf45505a9ae8139c35693f158fbf27c9e0191247a866dedba43489a4d0 WatchSource:0}: Error finding container 14a8d7bf45505a9ae8139c35693f158fbf27c9e0191247a866dedba43489a4d0: Status 404 returned error can't find the container with id 14a8d7bf45505a9ae8139c35693f158fbf27c9e0191247a866dedba43489a4d0
Sep 08 15:34:11.437529 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:11.437478    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt" event={"ID":"3bde229e-169e-4691-9941-b15f266e47a2","Type":"ContainerStarted","Data":"7af3c7a24fbaf5e054ad67ac9a36582197814e3a4ddb77394b27e67bbb59f084"}
Sep 08 15:34:11.437934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:11.437536    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt" event={"ID":"3bde229e-169e-4691-9941-b15f266e47a2","Type":"ContainerStarted","Data":"14a8d7bf45505a9ae8139c35693f158fbf27c9e0191247a866dedba43489a4d0"}
Sep 08 15:34:11.458354 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:11.458310    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt" podStartSLOduration=2.458294826 podStartE2EDuration="2.458294826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:11.455616957 +0000 UTC m=+1499.541566995" watchObservedRunningTime="2026-09-08 15:34:11.458294826 +0000 UTC m=+1499.544244868"
Sep 08 15:34:12.422891 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:12.422862    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:34:12.429063 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:12.429037    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:34:20.257144 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:20.257100    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:20.257665 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:20.257175    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:20.269705 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:20.269680    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:20.496691 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:20.496663    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:34:25.944823 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:25.944786    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:34:38.350323 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:38.350245    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"]
Sep 08 15:34:38.372948 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:38.372916    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:40.024853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:40.024822    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"]
Sep 08 15:34:40.049696 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:40.049669    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:43.363079 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:43.363046    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:34:48.150418 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:48.150382    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:34:53.369673 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.369614    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk" podUID="4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd" containerName="main" containerID="cri-o://e12015454b7cb9868882ab67f2d19c06470b05ab259a99a78aa76c1c945fb9ec" gracePeriod=45
Sep 08 15:34:53.620785 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.620710    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:34:53.672773 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.672738    2575 generic.go:356] "Generic (PLEG): container finished" podID="4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd" containerID="e12015454b7cb9868882ab67f2d19c06470b05ab259a99a78aa76c1c945fb9ec" exitCode=0
Sep 08 15:34:53.672773 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.672775    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"
Sep 08 15:34:53.672946 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.672797    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk" event={"ID":"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd","Type":"ContainerDied","Data":"e12015454b7cb9868882ab67f2d19c06470b05ab259a99a78aa76c1c945fb9ec"}
Sep 08 15:34:53.672946 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.672850    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk" event={"ID":"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd","Type":"ContainerDied","Data":"e32be9fd902b759880f1d5461a7338fdcda7fafe692253b5e5990ff8ce724457"}
Sep 08 15:34:53.672946 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.672869    2575 scope.go:117] "RemoveContainer" containerID="e12015454b7cb9868882ab67f2d19c06470b05ab259a99a78aa76c1c945fb9ec"
Sep 08 15:34:53.681962 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.681941    2575 scope.go:117] "RemoveContainer" containerID="e12015454b7cb9868882ab67f2d19c06470b05ab259a99a78aa76c1c945fb9ec"
Sep 08 15:34:53.682223 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:34:53.682204    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e12015454b7cb9868882ab67f2d19c06470b05ab259a99a78aa76c1c945fb9ec\": container with ID starting with e12015454b7cb9868882ab67f2d19c06470b05ab259a99a78aa76c1c945fb9ec not found: ID does not exist" containerID="e12015454b7cb9868882ab67f2d19c06470b05ab259a99a78aa76c1c945fb9ec"
Sep 08 15:34:53.682297 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.682235    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e12015454b7cb9868882ab67f2d19c06470b05ab259a99a78aa76c1c945fb9ec"} err="failed to get container status \"e12015454b7cb9868882ab67f2d19c06470b05ab259a99a78aa76c1c945fb9ec\": rpc error: code = NotFound desc = could not find container \"e12015454b7cb9868882ab67f2d19c06470b05ab259a99a78aa76c1c945fb9ec\": container with ID starting with e12015454b7cb9868882ab67f2d19c06470b05ab259a99a78aa76c1c945fb9ec not found: ID does not exist"
Sep 08 15:34:53.763994 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.763976    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-home\") pod \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") "
Sep 08 15:34:53.764097 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.764003    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-tls-certs\") pod \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") "
Sep 08 15:34:53.764097 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.764056    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-model-cache\") pod \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") "
Sep 08 15:34:53.764172 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.764103    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-dshm\") pod \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") "
Sep 08 15:34:53.764219 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.764166    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44zcg\" (UniqueName: \"kubernetes.io/projected/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-kube-api-access-44zcg\") pod \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") "
Sep 08 15:34:53.764219 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.764203    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-tmp-dir\") pod \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\" (UID: \"4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd\") "
Sep 08 15:34:53.764320 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.764235    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-home" (OuterVolumeSpecName: "home") pod "4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd" (UID: "4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:34:53.764320 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.764296    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd" (UID: "4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:34:53.764558 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.764537    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd" (UID: "4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:34:53.764654 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.764611    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:34:53.764654 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.764626    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:34:53.766238 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.766190    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-dshm" (OuterVolumeSpecName: "dshm") pod "4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd" (UID: "4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:34:53.766345 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.766266    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd" (UID: "4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:34:53.766345 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.766302    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-kube-api-access-44zcg" (OuterVolumeSpecName: "kube-api-access-44zcg") pod "4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd" (UID: "4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd"). InnerVolumeSpecName "kube-api-access-44zcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:34:53.865845 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.865822    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:34:53.865845 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.865843    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:34:53.865983 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.865852    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44zcg\" (UniqueName: \"kubernetes.io/projected/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-kube-api-access-44zcg\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:34:53.865983 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:53.865861    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:34:54.002175 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:54.002149    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"]
Sep 08 15:34:54.006671 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:54.006635    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2e082e/auth-test-model-2e082e-kserve-7d4dd4d48c-ssfnk"]
Sep 08 15:34:54.432021 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:54.431988    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd" path="/var/lib/kubelet/pods/4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd/volumes"
Sep 08 15:34:55.046200 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.046168    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc" podUID="a6895f35-f7d0-4dcb-aa11-c7c5e694afa9" containerName="main" containerID="cri-o://9001bac29292bfa2c853d8d0d413d673e0ae3baef7280541359fcfa6155c37cf" gracePeriod=45
Sep 08 15:34:55.295314 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.295291    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:34:55.480405 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.480375    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-tls-certs\") pod \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") "
Sep 08 15:34:55.480853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.480424    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-home\") pod \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") "
Sep 08 15:34:55.480853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.480467    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-tmp-dir\") pod \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") "
Sep 08 15:34:55.480853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.480526    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdnxx\" (UniqueName: \"kubernetes.io/projected/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-kube-api-access-qdnxx\") pod \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") "
Sep 08 15:34:55.480853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.480558    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-model-cache\") pod \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") "
Sep 08 15:34:55.480853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.480577    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-dshm\") pod \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\" (UID: \"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9\") "
Sep 08 15:34:55.480853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.480757    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a6895f35-f7d0-4dcb-aa11-c7c5e694afa9" (UID: "a6895f35-f7d0-4dcb-aa11-c7c5e694afa9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:34:55.480853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.480789    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-home" (OuterVolumeSpecName: "home") pod "a6895f35-f7d0-4dcb-aa11-c7c5e694afa9" (UID: "a6895f35-f7d0-4dcb-aa11-c7c5e694afa9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:34:55.480853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.480827    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-model-cache" (OuterVolumeSpecName: "model-cache") pod "a6895f35-f7d0-4dcb-aa11-c7c5e694afa9" (UID: "a6895f35-f7d0-4dcb-aa11-c7c5e694afa9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:34:55.482685 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.482664    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-kube-api-access-qdnxx" (OuterVolumeSpecName: "kube-api-access-qdnxx") pod "a6895f35-f7d0-4dcb-aa11-c7c5e694afa9" (UID: "a6895f35-f7d0-4dcb-aa11-c7c5e694afa9"). InnerVolumeSpecName "kube-api-access-qdnxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:34:55.482814 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.482706    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a6895f35-f7d0-4dcb-aa11-c7c5e694afa9" (UID: "a6895f35-f7d0-4dcb-aa11-c7c5e694afa9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:34:55.482814 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.482746    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-dshm" (OuterVolumeSpecName: "dshm") pod "a6895f35-f7d0-4dcb-aa11-c7c5e694afa9" (UID: "a6895f35-f7d0-4dcb-aa11-c7c5e694afa9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:34:55.581637 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.581610    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:34:55.581637 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.581633    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:34:55.581768 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.581647    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:34:55.581768 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.581658    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdnxx\" (UniqueName: \"kubernetes.io/projected/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-kube-api-access-qdnxx\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:34:55.581768 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.581670    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:34:55.581768 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.581682    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:34:55.686945 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.686915    2575 generic.go:356] "Generic (PLEG): container finished" podID="a6895f35-f7d0-4dcb-aa11-c7c5e694afa9" containerID="9001bac29292bfa2c853d8d0d413d673e0ae3baef7280541359fcfa6155c37cf" exitCode=0
Sep 08 15:34:55.687045 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.686957    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"
Sep 08 15:34:55.687045 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.687004    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc" event={"ID":"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9","Type":"ContainerDied","Data":"9001bac29292bfa2c853d8d0d413d673e0ae3baef7280541359fcfa6155c37cf"}
Sep 08 15:34:55.687174 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.687049    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc" event={"ID":"a6895f35-f7d0-4dcb-aa11-c7c5e694afa9","Type":"ContainerDied","Data":"e002c4259399cc373e7d0b5509e855e797f9ae68d07d6e4789c3ea39d7eb3499"}
Sep 08 15:34:55.687174 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.687067    2575 scope.go:117] "RemoveContainer" containerID="9001bac29292bfa2c853d8d0d413d673e0ae3baef7280541359fcfa6155c37cf"
Sep 08 15:34:55.696228 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.696210    2575 scope.go:117] "RemoveContainer" containerID="9001bac29292bfa2c853d8d0d413d673e0ae3baef7280541359fcfa6155c37cf"
Sep 08 15:34:55.696504 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:34:55.696474    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9001bac29292bfa2c853d8d0d413d673e0ae3baef7280541359fcfa6155c37cf\": container with ID starting with 9001bac29292bfa2c853d8d0d413d673e0ae3baef7280541359fcfa6155c37cf not found: ID does not exist" containerID="9001bac29292bfa2c853d8d0d413d673e0ae3baef7280541359fcfa6155c37cf"
Sep 08 15:34:55.696571 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.696514    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9001bac29292bfa2c853d8d0d413d673e0ae3baef7280541359fcfa6155c37cf"} err="failed to get container status \"9001bac29292bfa2c853d8d0d413d673e0ae3baef7280541359fcfa6155c37cf\": rpc error: code = NotFound desc = could not find container \"9001bac29292bfa2c853d8d0d413d673e0ae3baef7280541359fcfa6155c37cf\": container with ID starting with 9001bac29292bfa2c853d8d0d413d673e0ae3baef7280541359fcfa6155c37cf not found: ID does not exist"
Sep 08 15:34:55.710609 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.710579    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"]
Sep 08 15:34:55.714409 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:55.714389    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/auth-test-model-bc9cdf-kserve-fd77b45c9-687tc"]
Sep 08 15:34:56.432406 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:34:56.432378    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6895f35-f7d0-4dcb-aa11-c7c5e694afa9" path="/var/lib/kubelet/pods/a6895f35-f7d0-4dcb-aa11-c7c5e694afa9/volumes"
Sep 08 15:35:05.837951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.837921    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"]
Sep 08 15:35:05.838945 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.838919    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd" containerName="main"
Sep 08 15:35:05.838945 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.838942    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd" containerName="main"
Sep 08 15:35:05.839133 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.838953    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6895f35-f7d0-4dcb-aa11-c7c5e694afa9" containerName="main"
Sep 08 15:35:05.839133 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.838959    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6895f35-f7d0-4dcb-aa11-c7c5e694afa9" containerName="main"
Sep 08 15:35:05.839311 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.839279    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6895f35-f7d0-4dcb-aa11-c7c5e694afa9" containerName="main"
Sep 08 15:35:05.839439 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.839324    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e3fb9f1-cfc4-48c6-b67f-f206a17e1cfd" containerName="main"
Sep 08 15:35:05.849035 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.849007    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.852120 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.851941    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-bc9cdf\"/\"openshift-service-ca.crt\""
Sep 08 15:35:05.852250 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.852181    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-bc9cdf\"/\"sub-test-model-bc9cdf-kserve-self-signed-certs\""
Sep 08 15:35:05.852975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.852952    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-bc9cdf\"/\"kube-root-ca.crt\""
Sep 08 15:35:05.853100 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.852952    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-bc9cdf\"/\"default-dockercfg-xpclp\""
Sep 08 15:35:05.853242 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.853222    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"]
Sep 08 15:35:05.861444 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.861421    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:05.872948 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.872920    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-dshm\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.873045 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.872951    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-model-cache\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.873045 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.872989    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rctf8\" (UniqueName: \"kubernetes.io/projected/1cab2d97-430d-4920-b4b7-cfcf641e39dd-kube-api-access-rctf8\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.873122 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.873075    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-tmp-dir\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.873159 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.873119    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-home\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.873221 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.873204    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cab2d97-430d-4920-b4b7-cfcf641e39dd-tls-certs\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.974561 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.974532    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cab2d97-430d-4920-b4b7-cfcf641e39dd-tls-certs\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.974654 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.974598    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-dshm\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.974654 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.974614    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-model-cache\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.974654 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.974637    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rctf8\" (UniqueName: \"kubernetes.io/projected/1cab2d97-430d-4920-b4b7-cfcf641e39dd-kube-api-access-rctf8\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.974821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.974665    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-tmp-dir\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.974821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.974694    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-home\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.975142 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.975117    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-home\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.975231 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.975140    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-model-cache\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.975231 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.975188    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-tmp-dir\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.976788 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.976769    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-dshm\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.976967 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.976951    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cab2d97-430d-4920-b4b7-cfcf641e39dd-tls-certs\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:05.983564 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:05.983544    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rctf8\" (UniqueName: \"kubernetes.io/projected/1cab2d97-430d-4920-b4b7-cfcf641e39dd-kube-api-access-rctf8\") pod \"sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:06.159722 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:06.159694    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:06.433721 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:06.433515    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"]
Sep 08 15:35:06.436700 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:35:06.436675    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1cab2d97_430d_4920_b4b7_cfcf641e39dd.slice/crio-c172c3d2ce953a5134ae28ae3ca52a2d2b9566272eaedaed78a467f36dfce441 WatchSource:0}: Error finding container c172c3d2ce953a5134ae28ae3ca52a2d2b9566272eaedaed78a467f36dfce441: Status 404 returned error can't find the container with id c172c3d2ce953a5134ae28ae3ca52a2d2b9566272eaedaed78a467f36dfce441
Sep 08 15:35:06.748301 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:06.748221    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss" event={"ID":"1cab2d97-430d-4920-b4b7-cfcf641e39dd","Type":"ContainerStarted","Data":"a36eb83912845e7f847abec3fdb9a807bd800bcd0172df3a21c9d6c2b5a78e13"}
Sep 08 15:35:06.748301 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:06.748265    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss" event={"ID":"1cab2d97-430d-4920-b4b7-cfcf641e39dd","Type":"ContainerStarted","Data":"c172c3d2ce953a5134ae28ae3ca52a2d2b9566272eaedaed78a467f36dfce441"}
Sep 08 15:35:06.767604 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:06.767560    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss" podStartSLOduration=1.7675447210000002 podStartE2EDuration="1.767544721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:35:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:35:06.766612438 +0000 UTC m=+1554.852562492" watchObservedRunningTime="2026-09-08 15:35:06.767544721 +0000 UTC m=+1554.853494763"
Sep 08 15:35:10.544861 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:10.544821    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:12.417068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.417037    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"]
Sep 08 15:35:12.417475 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.417242    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx" podUID="bc73d9eb-d418-440a-95ed-2aa2706172f0" containerName="maas-api" containerID="cri-o://da0aa64ba2ace139a11f931067fb1a744d7c1743f3ed4d1d9be6f68045400b61" gracePeriod=30
Sep 08 15:35:12.467208 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.467167    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:12.489221 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.489196    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:35:12.616407 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.616369    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:12.683757 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.683735    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:35:12.718948 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.718925    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:12.734405 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.734383    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc73d9eb-d418-440a-95ed-2aa2706172f0-maas-api-tls\") pod \"bc73d9eb-d418-440a-95ed-2aa2706172f0\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") "
Sep 08 15:35:12.734515 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.734422    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc73d9eb-d418-440a-95ed-2aa2706172f0-metrics-certs\") pod \"bc73d9eb-d418-440a-95ed-2aa2706172f0\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") "
Sep 08 15:35:12.734591 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.734575    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc73d9eb-d418-440a-95ed-2aa2706172f0-openshift-service-ca\") pod \"bc73d9eb-d418-440a-95ed-2aa2706172f0\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") "
Sep 08 15:35:12.734675 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.734661    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x9hp\" (UniqueName: \"kubernetes.io/projected/bc73d9eb-d418-440a-95ed-2aa2706172f0-kube-api-access-2x9hp\") pod \"bc73d9eb-d418-440a-95ed-2aa2706172f0\" (UID: \"bc73d9eb-d418-440a-95ed-2aa2706172f0\") "
Sep 08 15:35:12.734968 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.734940    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc73d9eb-d418-440a-95ed-2aa2706172f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bc73d9eb-d418-440a-95ed-2aa2706172f0" (UID: "bc73d9eb-d418-440a-95ed-2aa2706172f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:35:12.736443 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.736416    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc73d9eb-d418-440a-95ed-2aa2706172f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bc73d9eb-d418-440a-95ed-2aa2706172f0" (UID: "bc73d9eb-d418-440a-95ed-2aa2706172f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:12.736960 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.736933    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc73d9eb-d418-440a-95ed-2aa2706172f0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bc73d9eb-d418-440a-95ed-2aa2706172f0" (UID: "bc73d9eb-d418-440a-95ed-2aa2706172f0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:12.737038 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.736970    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc73d9eb-d418-440a-95ed-2aa2706172f0-kube-api-access-2x9hp" (OuterVolumeSpecName: "kube-api-access-2x9hp") pod "bc73d9eb-d418-440a-95ed-2aa2706172f0" (UID: "bc73d9eb-d418-440a-95ed-2aa2706172f0"). InnerVolumeSpecName "kube-api-access-2x9hp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:35:12.782542 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.782511    2575 generic.go:356] "Generic (PLEG): container finished" podID="bc73d9eb-d418-440a-95ed-2aa2706172f0" containerID="da0aa64ba2ace139a11f931067fb1a744d7c1743f3ed4d1d9be6f68045400b61" exitCode=0
Sep 08 15:35:12.782542 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.782539    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"
Sep 08 15:35:12.782678 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.782586    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx" event={"ID":"bc73d9eb-d418-440a-95ed-2aa2706172f0","Type":"ContainerDied","Data":"da0aa64ba2ace139a11f931067fb1a744d7c1743f3ed4d1d9be6f68045400b61"}
Sep 08 15:35:12.782678 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.782634    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx" event={"ID":"bc73d9eb-d418-440a-95ed-2aa2706172f0","Type":"ContainerDied","Data":"3b6b30874d47e7faad57a0fd9c43221682feaab7255555ea854ea6c0ee37ed6e"}
Sep 08 15:35:12.782678 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.782655    2575 scope.go:117] "RemoveContainer" containerID="da0aa64ba2ace139a11f931067fb1a744d7c1743f3ed4d1d9be6f68045400b61"
Sep 08 15:35:12.791984 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.791968    2575 scope.go:117] "RemoveContainer" containerID="da0aa64ba2ace139a11f931067fb1a744d7c1743f3ed4d1d9be6f68045400b61"
Sep 08 15:35:12.792251 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:35:12.792232    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da0aa64ba2ace139a11f931067fb1a744d7c1743f3ed4d1d9be6f68045400b61\": container with ID starting with da0aa64ba2ace139a11f931067fb1a744d7c1743f3ed4d1d9be6f68045400b61 not found: ID does not exist" containerID="da0aa64ba2ace139a11f931067fb1a744d7c1743f3ed4d1d9be6f68045400b61"
Sep 08 15:35:12.792322 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.792262    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da0aa64ba2ace139a11f931067fb1a744d7c1743f3ed4d1d9be6f68045400b61"} err="failed to get container status \"da0aa64ba2ace139a11f931067fb1a744d7c1743f3ed4d1d9be6f68045400b61\": rpc error: code = NotFound desc = could not find container \"da0aa64ba2ace139a11f931067fb1a744d7c1743f3ed4d1d9be6f68045400b61\": container with ID starting with da0aa64ba2ace139a11f931067fb1a744d7c1743f3ed4d1d9be6f68045400b61 not found: ID does not exist"
Sep 08 15:35:12.811774 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.811756    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"]
Sep 08 15:35:12.818317 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.818296    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ae34c54a-c5865764-9lhgx"]
Sep 08 15:35:12.819457 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.819437    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:12.835715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.835691    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc73d9eb-d418-440a-95ed-2aa2706172f0-openshift-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:35:12.835715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.835713    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x9hp\" (UniqueName: \"kubernetes.io/projected/bc73d9eb-d418-440a-95ed-2aa2706172f0-kube-api-access-2x9hp\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:35:12.835828 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.835723    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc73d9eb-d418-440a-95ed-2aa2706172f0-maas-api-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:35:12.835828 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.835732    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc73d9eb-d418-440a-95ed-2aa2706172f0-metrics-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:35:12.898297 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.898269    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:12.899350 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:12.899331    2575 scope.go:117] "RemoveContainer" containerID="556c421090e61285fdf83ed2eb87391b3320895c2630f73501f589bba4262552"
Sep 08 15:35:14.434904 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:14.434869    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc73d9eb-d418-440a-95ed-2aa2706172f0" path="/var/lib/kubelet/pods/bc73d9eb-d418-440a-95ed-2aa2706172f0/volumes"
Sep 08 15:35:16.139971 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:16.139931    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:16.160475 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:16.160438    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:16.160731 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:16.160714    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:16.172712 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:16.172690    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:16.813697 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:16.813670    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:35:21.128800 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:21.128764    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"]
Sep 08 15:35:21.224207 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:21.224172    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:28.437101 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:28.437068    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:35:36.148535 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.148470    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt" podUID="3bde229e-169e-4691-9941-b15f266e47a2" containerName="main" containerID="cri-o://7af3c7a24fbaf5e054ad67ac9a36582197814e3a4ddb77394b27e67bbb59f084" gracePeriod=45
Sep 08 15:35:36.399704 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.399647    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:35:36.551453 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.551421    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-model-cache\") pod \"3bde229e-169e-4691-9941-b15f266e47a2\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") "
Sep 08 15:35:36.551655 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.551467    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-home\") pod \"3bde229e-169e-4691-9941-b15f266e47a2\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") "
Sep 08 15:35:36.551655 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.551520    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-dshm\") pod \"3bde229e-169e-4691-9941-b15f266e47a2\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") "
Sep 08 15:35:36.551655 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.551610    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99cp2\" (UniqueName: \"kubernetes.io/projected/3bde229e-169e-4691-9941-b15f266e47a2-kube-api-access-99cp2\") pod \"3bde229e-169e-4691-9941-b15f266e47a2\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") "
Sep 08 15:35:36.551655 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.551629    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bde229e-169e-4691-9941-b15f266e47a2-tls-certs\") pod \"3bde229e-169e-4691-9941-b15f266e47a2\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") "
Sep 08 15:35:36.551655 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.551654    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-tmp-dir\") pod \"3bde229e-169e-4691-9941-b15f266e47a2\" (UID: \"3bde229e-169e-4691-9941-b15f266e47a2\") "
Sep 08 15:35:36.551927 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.551751    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-model-cache" (OuterVolumeSpecName: "model-cache") pod "3bde229e-169e-4691-9941-b15f266e47a2" (UID: "3bde229e-169e-4691-9941-b15f266e47a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:35:36.551927 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.551796    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-home" (OuterVolumeSpecName: "home") pod "3bde229e-169e-4691-9941-b15f266e47a2" (UID: "3bde229e-169e-4691-9941-b15f266e47a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:35:36.552054 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.552023    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3bde229e-169e-4691-9941-b15f266e47a2" (UID: "3bde229e-169e-4691-9941-b15f266e47a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:35:36.552152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.552135    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:35:36.552209 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.552158    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:35:36.553759 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.553737    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bde229e-169e-4691-9941-b15f266e47a2-kube-api-access-99cp2" (OuterVolumeSpecName: "kube-api-access-99cp2") pod "3bde229e-169e-4691-9941-b15f266e47a2" (UID: "3bde229e-169e-4691-9941-b15f266e47a2"). InnerVolumeSpecName "kube-api-access-99cp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:35:36.553823 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.553749    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bde229e-169e-4691-9941-b15f266e47a2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3bde229e-169e-4691-9941-b15f266e47a2" (UID: "3bde229e-169e-4691-9941-b15f266e47a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:36.553823 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.553803    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-dshm" (OuterVolumeSpecName: "dshm") pod "3bde229e-169e-4691-9941-b15f266e47a2" (UID: "3bde229e-169e-4691-9941-b15f266e47a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:35:36.653432 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.653370    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:35:36.653432 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.653393    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bde229e-169e-4691-9941-b15f266e47a2-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:35:36.653432 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.653403    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99cp2\" (UniqueName: \"kubernetes.io/projected/3bde229e-169e-4691-9941-b15f266e47a2-kube-api-access-99cp2\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:35:36.653432 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.653413    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bde229e-169e-4691-9941-b15f266e47a2-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:35:36.922639 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.922481    2575 generic.go:356] "Generic (PLEG): container finished" podID="3bde229e-169e-4691-9941-b15f266e47a2" containerID="7af3c7a24fbaf5e054ad67ac9a36582197814e3a4ddb77394b27e67bbb59f084" exitCode=0
Sep 08 15:35:36.922639 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.922541    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt" event={"ID":"3bde229e-169e-4691-9941-b15f266e47a2","Type":"ContainerDied","Data":"7af3c7a24fbaf5e054ad67ac9a36582197814e3a4ddb77394b27e67bbb59f084"}
Sep 08 15:35:36.922639 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.922585    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"
Sep 08 15:35:36.922639 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.922626    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt" event={"ID":"3bde229e-169e-4691-9941-b15f266e47a2","Type":"ContainerDied","Data":"14a8d7bf45505a9ae8139c35693f158fbf27c9e0191247a866dedba43489a4d0"}
Sep 08 15:35:36.922965 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.922656    2575 scope.go:117] "RemoveContainer" containerID="7af3c7a24fbaf5e054ad67ac9a36582197814e3a4ddb77394b27e67bbb59f084"
Sep 08 15:35:36.933149 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.933128    2575 scope.go:117] "RemoveContainer" containerID="7af3c7a24fbaf5e054ad67ac9a36582197814e3a4ddb77394b27e67bbb59f084"
Sep 08 15:35:36.933430 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:35:36.933414    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7af3c7a24fbaf5e054ad67ac9a36582197814e3a4ddb77394b27e67bbb59f084\": container with ID starting with 7af3c7a24fbaf5e054ad67ac9a36582197814e3a4ddb77394b27e67bbb59f084 not found: ID does not exist" containerID="7af3c7a24fbaf5e054ad67ac9a36582197814e3a4ddb77394b27e67bbb59f084"
Sep 08 15:35:36.933485 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.933438    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7af3c7a24fbaf5e054ad67ac9a36582197814e3a4ddb77394b27e67bbb59f084"} err="failed to get container status \"7af3c7a24fbaf5e054ad67ac9a36582197814e3a4ddb77394b27e67bbb59f084\": rpc error: code = NotFound desc = could not find container \"7af3c7a24fbaf5e054ad67ac9a36582197814e3a4ddb77394b27e67bbb59f084\": container with ID starting with 7af3c7a24fbaf5e054ad67ac9a36582197814e3a4ddb77394b27e67bbb59f084 not found: ID does not exist"
Sep 08 15:35:36.946815 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.946792    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"]
Sep 08 15:35:36.950865 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:36.950845    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ae34c54a/e2e-lifecycle-model-ae34c54a-kserve-68dc77985f-fv8jt"]
Sep 08 15:35:38.431875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:38.431844    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bde229e-169e-4691-9941-b15f266e47a2" path="/var/lib/kubelet/pods/3bde229e-169e-4691-9941-b15f266e47a2/volumes"
Sep 08 15:35:42.154877 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.154844    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7852-60b5d3"]
Sep 08 15:35:42.155385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.155271    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc73d9eb-d418-440a-95ed-2aa2706172f0" containerName="maas-api"
Sep 08 15:35:42.155385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.155284    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc73d9eb-d418-440a-95ed-2aa2706172f0" containerName="maas-api"
Sep 08 15:35:42.155385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.155299    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bde229e-169e-4691-9941-b15f266e47a2" containerName="main"
Sep 08 15:35:42.155385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.155305    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bde229e-169e-4691-9941-b15f266e47a2" containerName="main"
Sep 08 15:35:42.155385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.155379    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bde229e-169e-4691-9941-b15f266e47a2" containerName="main"
Sep 08 15:35:42.155385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.155387    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc73d9eb-d418-440a-95ed-2aa2706172f0" containerName="maas-api"
Sep 08 15:35:42.160065 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.160048    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7852-60b5d3"
Sep 08 15:35:42.163971 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.163953    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kvpjx\""
Sep 08 15:35:42.175337 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.175312    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7852-60b5d3"]
Sep 08 15:35:42.208858 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.208833    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hgbf\" (UniqueName: \"kubernetes.io/projected/0d1d9e93-ecc1-4f02-95aa-83614f151dc5-kube-api-access-9hgbf\") pod \"mt-curl-7852-60b5d3\" (UID: \"0d1d9e93-ecc1-4f02-95aa-83614f151dc5\") " pod="opendatahub/mt-curl-7852-60b5d3"
Sep 08 15:35:42.310076 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.310049    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hgbf\" (UniqueName: \"kubernetes.io/projected/0d1d9e93-ecc1-4f02-95aa-83614f151dc5-kube-api-access-9hgbf\") pod \"mt-curl-7852-60b5d3\" (UID: \"0d1d9e93-ecc1-4f02-95aa-83614f151dc5\") " pod="opendatahub/mt-curl-7852-60b5d3"
Sep 08 15:35:42.321015 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.320994    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hgbf\" (UniqueName: \"kubernetes.io/projected/0d1d9e93-ecc1-4f02-95aa-83614f151dc5-kube-api-access-9hgbf\") pod \"mt-curl-7852-60b5d3\" (UID: \"0d1d9e93-ecc1-4f02-95aa-83614f151dc5\") " pod="opendatahub/mt-curl-7852-60b5d3"
Sep 08 15:35:42.469279 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.469219    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7852-60b5d3"
Sep 08 15:35:42.752935 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.746812    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7852-60b5d3"]
Sep 08 15:35:42.754788 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:35:42.754752    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d1d9e93_ecc1_4f02_95aa_83614f151dc5.slice/crio-2462ccf289c39dda3ef610fc55409faebf9d578faf427ae456b55990ade47cc4 WatchSource:0}: Error finding container 2462ccf289c39dda3ef610fc55409faebf9d578faf427ae456b55990ade47cc4: Status 404 returned error can't find the container with id 2462ccf289c39dda3ef610fc55409faebf9d578faf427ae456b55990ade47cc4
Sep 08 15:35:42.957464 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:42.957426    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7852-60b5d3" event={"ID":"0d1d9e93-ecc1-4f02-95aa-83614f151dc5","Type":"ContainerStarted","Data":"2462ccf289c39dda3ef610fc55409faebf9d578faf427ae456b55990ade47cc4"}
Sep 08 15:35:43.963895 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:43.963852    2575 generic.go:356] "Generic (PLEG): container finished" podID="0d1d9e93-ecc1-4f02-95aa-83614f151dc5" containerID="cf3f17f4a86aa8451d8d983e4b701aa186dc75d9ac52c29eccf847e3b799724d" exitCode=0
Sep 08 15:35:43.964264 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:43.963946    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7852-60b5d3" event={"ID":"0d1d9e93-ecc1-4f02-95aa-83614f151dc5","Type":"ContainerDied","Data":"cf3f17f4a86aa8451d8d983e4b701aa186dc75d9ac52c29eccf847e3b799724d"}
Sep 08 15:35:45.101438 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:45.101416    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7852-60b5d3"
Sep 08 15:35:45.137240 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:45.137216    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7852-60b5d3_0d1d9e93-ecc1-4f02-95aa-83614f151dc5/mt-curl-7852-60b5d3/0.log"
Sep 08 15:35:45.138390 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:45.138372    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hgbf\" (UniqueName: \"kubernetes.io/projected/0d1d9e93-ecc1-4f02-95aa-83614f151dc5-kube-api-access-9hgbf\") pod \"0d1d9e93-ecc1-4f02-95aa-83614f151dc5\" (UID: \"0d1d9e93-ecc1-4f02-95aa-83614f151dc5\") "
Sep 08 15:35:45.140330 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:45.140306    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d1d9e93-ecc1-4f02-95aa-83614f151dc5-kube-api-access-9hgbf" (OuterVolumeSpecName: "kube-api-access-9hgbf") pod "0d1d9e93-ecc1-4f02-95aa-83614f151dc5" (UID: "0d1d9e93-ecc1-4f02-95aa-83614f151dc5"). InnerVolumeSpecName "kube-api-access-9hgbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:35:45.181074 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:45.181046    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7852-60b5d3"]
Sep 08 15:35:45.182888 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:45.182867    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7852-60b5d3"]
Sep 08 15:35:45.239457 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:45.239404    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hgbf\" (UniqueName: \"kubernetes.io/projected/0d1d9e93-ecc1-4f02-95aa-83614f151dc5-kube-api-access-9hgbf\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:35:45.976712 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:45.976684    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7852-60b5d3"
Sep 08 15:35:45.976712 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:45.976714    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2462ccf289c39dda3ef610fc55409faebf9d578faf427ae456b55990ade47cc4"
Sep 08 15:35:45.987355 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:35:45.987331    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7852-60b5d3\" is forbidden: User \"system:node:ip-10-0-132-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-76.ec2.internal' and this object" podUID="0d1d9e93-ecc1-4f02-95aa-83614f151dc5" pod="opendatahub/mt-curl-7852-60b5d3"
Sep 08 15:35:46.432012 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:46.431982    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d1d9e93-ecc1-4f02-95aa-83614f151dc5" path="/var/lib/kubelet/pods/0d1d9e93-ecc1-4f02-95aa-83614f151dc5/volumes"
Sep 08 15:35:48.144741 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:48.144698    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:35:50.307042 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:50.307004    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:51.396935 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:51.396895    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"]
Sep 08 15:35:52.562540 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:52.562468    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:58.215419 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:58.215380    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:35:58.804026 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:35:58.803988    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:03.060167 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:03.060131    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:36:05.178885 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.178786    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:05.261039 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.261009    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"]
Sep 08 15:36:05.261436 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.261424    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d1d9e93-ecc1-4f02-95aa-83614f151dc5" containerName="mt-curl-7852-60b5d3"
Sep 08 15:36:05.261505 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.261437    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d1d9e93-ecc1-4f02-95aa-83614f151dc5" containerName="mt-curl-7852-60b5d3"
Sep 08 15:36:05.261576 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.261557    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d1d9e93-ecc1-4f02-95aa-83614f151dc5" containerName="mt-curl-7852-60b5d3"
Sep 08 15:36:05.264764 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.264749    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.268773 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.268750    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2e082e\"/\"default-dockercfg-bq58p\""
Sep 08 15:36:05.268898 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.268814    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2e082e\"/\"rate-test-model-2e082e-kserve-self-signed-certs\""
Sep 08 15:36:05.269053 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.269037    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2e082e\"/\"openshift-service-ca.crt\""
Sep 08 15:36:05.269114 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.269087    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2e082e\"/\"kube-root-ca.crt\""
Sep 08 15:36:05.277221 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.277197    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"]
Sep 08 15:36:05.282352 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.282325    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:05.347427 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.347395    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-tmp-dir\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.347604 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.347450    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-dshm\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.347604 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.347534    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw9ht\" (UniqueName: \"kubernetes.io/projected/a4eafea6-a344-4962-80d5-59000507b236-kube-api-access-gw9ht\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.347604 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.347580    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4eafea6-a344-4962-80d5-59000507b236-tls-certs\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.347713 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.347622    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-model-cache\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.347713 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.347656    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-home\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.449085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.448992    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw9ht\" (UniqueName: \"kubernetes.io/projected/a4eafea6-a344-4962-80d5-59000507b236-kube-api-access-gw9ht\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.449085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.449059    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4eafea6-a344-4962-80d5-59000507b236-tls-certs\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.449274 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.449125    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-model-cache\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.449274 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.449162    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-home\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.449274 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.449214    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-tmp-dir\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.449274 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.449268    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-dshm\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.449656 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.449630    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-tmp-dir\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.449755 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.449644    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-model-cache\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.449755 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.449689    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-home\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.451389 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.451367    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-dshm\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.451504 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.451462    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4eafea6-a344-4962-80d5-59000507b236-tls-certs\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.459017 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.458995    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw9ht\" (UniqueName: \"kubernetes.io/projected/a4eafea6-a344-4962-80d5-59000507b236-kube-api-access-gw9ht\") pod \"rate-test-model-2e082e-kserve-76bc465b95-b29fl\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") " pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.576211 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.576175    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:05.846961 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:05.846934    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"]
Sep 08 15:36:05.848306 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:36:05.848282    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4eafea6_a344_4962_80d5_59000507b236.slice/crio-83e618936326c1dc7ebc5567f4754bd0f584227e89b7b13c2458e041dabc79f6 WatchSource:0}: Error finding container 83e618936326c1dc7ebc5567f4754bd0f584227e89b7b13c2458e041dabc79f6: Status 404 returned error can't find the container with id 83e618936326c1dc7ebc5567f4754bd0f584227e89b7b13c2458e041dabc79f6
Sep 08 15:36:06.084387 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.084351    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl" event={"ID":"a4eafea6-a344-4962-80d5-59000507b236","Type":"ContainerStarted","Data":"d42f47329b0f2eaaaf3b348196038d85d50609e8977278df9260ad30d813f74a"}
Sep 08 15:36:06.084387 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.084396    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl" event={"ID":"a4eafea6-a344-4962-80d5-59000507b236","Type":"ContainerStarted","Data":"83e618936326c1dc7ebc5567f4754bd0f584227e89b7b13c2458e041dabc79f6"}
Sep 08 15:36:06.103963 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.103910    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl" podStartSLOduration=1.103894862 podStartE2EDuration="1.103894862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:36:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:36:06.102416562 +0000 UTC m=+1614.188366636" watchObservedRunningTime="2026-09-08 15:36:06.103894862 +0000 UTC m=+1614.189844902"
Sep 08 15:36:06.415608 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.415516    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss" podUID="1cab2d97-430d-4920-b4b7-cfcf641e39dd" containerName="main" containerID="cri-o://a36eb83912845e7f847abec3fdb9a807bd800bcd0172df3a21c9d6c2b5a78e13" gracePeriod=45
Sep 08 15:36:06.656894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.656866    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:36:06.762395 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.762310    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cab2d97-430d-4920-b4b7-cfcf641e39dd-tls-certs\") pod \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") "
Sep 08 15:36:06.762395 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.762371    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rctf8\" (UniqueName: \"kubernetes.io/projected/1cab2d97-430d-4920-b4b7-cfcf641e39dd-kube-api-access-rctf8\") pod \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") "
Sep 08 15:36:06.762609 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.762410    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-dshm\") pod \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") "
Sep 08 15:36:06.762609 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.762450    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-home\") pod \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") "
Sep 08 15:36:06.762609 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.762475    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-model-cache\") pod \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") "
Sep 08 15:36:06.762609 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.762516    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-tmp-dir\") pod \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\" (UID: \"1cab2d97-430d-4920-b4b7-cfcf641e39dd\") "
Sep 08 15:36:06.762827 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.762733    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-home" (OuterVolumeSpecName: "home") pod "1cab2d97-430d-4920-b4b7-cfcf641e39dd" (UID: "1cab2d97-430d-4920-b4b7-cfcf641e39dd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:36:06.762827 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.762749    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-model-cache" (OuterVolumeSpecName: "model-cache") pod "1cab2d97-430d-4920-b4b7-cfcf641e39dd" (UID: "1cab2d97-430d-4920-b4b7-cfcf641e39dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:36:06.762939 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.762899    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1cab2d97-430d-4920-b4b7-cfcf641e39dd" (UID: "1cab2d97-430d-4920-b4b7-cfcf641e39dd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:36:06.763049 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.763026    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:06.763049 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.763049    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:06.763221 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.763062    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:06.764504 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.764473    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cab2d97-430d-4920-b4b7-cfcf641e39dd-kube-api-access-rctf8" (OuterVolumeSpecName: "kube-api-access-rctf8") pod "1cab2d97-430d-4920-b4b7-cfcf641e39dd" (UID: "1cab2d97-430d-4920-b4b7-cfcf641e39dd"). InnerVolumeSpecName "kube-api-access-rctf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:36:06.764587 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.764526    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cab2d97-430d-4920-b4b7-cfcf641e39dd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1cab2d97-430d-4920-b4b7-cfcf641e39dd" (UID: "1cab2d97-430d-4920-b4b7-cfcf641e39dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:36:06.764587 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.764539    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-dshm" (OuterVolumeSpecName: "dshm") pod "1cab2d97-430d-4920-b4b7-cfcf641e39dd" (UID: "1cab2d97-430d-4920-b4b7-cfcf641e39dd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:36:06.864037 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.864001    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cab2d97-430d-4920-b4b7-cfcf641e39dd-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:06.864037 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.864032    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rctf8\" (UniqueName: \"kubernetes.io/projected/1cab2d97-430d-4920-b4b7-cfcf641e39dd-kube-api-access-rctf8\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:06.864037 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:06.864042    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cab2d97-430d-4920-b4b7-cfcf641e39dd-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:07.091545 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:07.091433    2575 generic.go:356] "Generic (PLEG): container finished" podID="1cab2d97-430d-4920-b4b7-cfcf641e39dd" containerID="a36eb83912845e7f847abec3fdb9a807bd800bcd0172df3a21c9d6c2b5a78e13" exitCode=0
Sep 08 15:36:07.091545 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:07.091479    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"
Sep 08 15:36:07.091545 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:07.091522    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss" event={"ID":"1cab2d97-430d-4920-b4b7-cfcf641e39dd","Type":"ContainerDied","Data":"a36eb83912845e7f847abec3fdb9a807bd800bcd0172df3a21c9d6c2b5a78e13"}
Sep 08 15:36:07.091812 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:07.091582    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss" event={"ID":"1cab2d97-430d-4920-b4b7-cfcf641e39dd","Type":"ContainerDied","Data":"c172c3d2ce953a5134ae28ae3ca52a2d2b9566272eaedaed78a467f36dfce441"}
Sep 08 15:36:07.091812 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:07.091613    2575 scope.go:117] "RemoveContainer" containerID="a36eb83912845e7f847abec3fdb9a807bd800bcd0172df3a21c9d6c2b5a78e13"
Sep 08 15:36:07.101636 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:07.101617    2575 scope.go:117] "RemoveContainer" containerID="a36eb83912845e7f847abec3fdb9a807bd800bcd0172df3a21c9d6c2b5a78e13"
Sep 08 15:36:07.101878 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:36:07.101858    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a36eb83912845e7f847abec3fdb9a807bd800bcd0172df3a21c9d6c2b5a78e13\": container with ID starting with a36eb83912845e7f847abec3fdb9a807bd800bcd0172df3a21c9d6c2b5a78e13 not found: ID does not exist" containerID="a36eb83912845e7f847abec3fdb9a807bd800bcd0172df3a21c9d6c2b5a78e13"
Sep 08 15:36:07.101940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:07.101890    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a36eb83912845e7f847abec3fdb9a807bd800bcd0172df3a21c9d6c2b5a78e13"} err="failed to get container status \"a36eb83912845e7f847abec3fdb9a807bd800bcd0172df3a21c9d6c2b5a78e13\": rpc error: code = NotFound desc = could not find container \"a36eb83912845e7f847abec3fdb9a807bd800bcd0172df3a21c9d6c2b5a78e13\": container with ID starting with a36eb83912845e7f847abec3fdb9a807bd800bcd0172df3a21c9d6c2b5a78e13 not found: ID does not exist"
Sep 08 15:36:07.115784 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:07.115751    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"]
Sep 08 15:36:07.120117 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:07.120093    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/sub-test-model-bc9cdf-kserve-7cd85b775c-9fmss"]
Sep 08 15:36:08.432151 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:08.432117    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cab2d97-430d-4920-b4b7-cfcf641e39dd" path="/var/lib/kubelet/pods/1cab2d97-430d-4920-b4b7-cfcf641e39dd/volumes"
Sep 08 15:36:11.571081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:11.571043    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:11.671803 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:11.671761    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:11.845215 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:11.845135    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"]
Sep 08 15:36:11.845366 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:11.845346    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp" podUID="cff875c0-c818-41c2-86a6-11f297288a35" containerName="istio-proxy" containerID="cri-o://51fee8dfc195e41de149ce80b861015fa873f70855e592bb3f9c67a86bd00280" gracePeriod=30
Sep 08 15:36:11.868067 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:11.868033    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:12.942374 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:12.942343    2575 scope.go:117] "RemoveContainer" containerID="ebea5f35db3c5dd241d7d8d16f07914dfd333d7c1139b7cb696867483f69e88f"
Sep 08 15:36:12.951220 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:12.951202    2575 scope.go:117] "RemoveContainer" containerID="e97f8fbc4f841eedd4ca7572a141e6a81ef1dd983645e7666032b8fa2fc83b8c"
Sep 08 15:36:12.961140 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:12.961118    2575 scope.go:117] "RemoveContainer" containerID="9cb9e69e65ea3a41ed29d025090f676b71587c3cd259d600720f09f17344ad31"
Sep 08 15:36:13.255819 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:13.255733    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:13.364771 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:13.364733    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:14.166868 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:14.166829    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:14.277278 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:14.277242    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:15.577276 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:15.577242    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:15.577694 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:15.577292    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:15.589757 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:15.589732    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:15.678729 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:15.678699    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:16.153715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:16.153683    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:36:17.097351 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.097324    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:36:17.150471 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.150391    2575 generic.go:356] "Generic (PLEG): container finished" podID="cff875c0-c818-41c2-86a6-11f297288a35" containerID="51fee8dfc195e41de149ce80b861015fa873f70855e592bb3f9c67a86bd00280" exitCode=0
Sep 08 15:36:17.150471 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.150441    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"
Sep 08 15:36:17.150678 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.150473    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp" event={"ID":"cff875c0-c818-41c2-86a6-11f297288a35","Type":"ContainerDied","Data":"51fee8dfc195e41de149ce80b861015fa873f70855e592bb3f9c67a86bd00280"}
Sep 08 15:36:17.150678 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.150583    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp" event={"ID":"cff875c0-c818-41c2-86a6-11f297288a35","Type":"ContainerDied","Data":"68dff083e2c0b4af152eddeae9f89730a0d94ea53d151bec657f382b7979073e"}
Sep 08 15:36:17.150678 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.150615    2575 scope.go:117] "RemoveContainer" containerID="51fee8dfc195e41de149ce80b861015fa873f70855e592bb3f9c67a86bd00280"
Sep 08 15:36:17.159795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.159771    2575 scope.go:117] "RemoveContainer" containerID="d7e637326aa7b35c8baaeef59fbe7576a0ffc3982c9ff1d6fcbdce015d4044c1"
Sep 08 15:36:17.159940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.159925    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cff875c0-c818-41c2-86a6-11f297288a35-istiod-ca-cert\") pod \"cff875c0-c818-41c2-86a6-11f297288a35\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") "
Sep 08 15:36:17.160000 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.159951    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-credential-socket\") pod \"cff875c0-c818-41c2-86a6-11f297288a35\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") "
Sep 08 15:36:17.160000 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.159974    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cff875c0-c818-41c2-86a6-11f297288a35-istio-token\") pod \"cff875c0-c818-41c2-86a6-11f297288a35\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") "
Sep 08 15:36:17.160101 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160016    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-istio-data\") pod \"cff875c0-c818-41c2-86a6-11f297288a35\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") "
Sep 08 15:36:17.160101 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160070    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqnbr\" (UniqueName: \"kubernetes.io/projected/cff875c0-c818-41c2-86a6-11f297288a35-kube-api-access-tqnbr\") pod \"cff875c0-c818-41c2-86a6-11f297288a35\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") "
Sep 08 15:36:17.160206 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160112    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cff875c0-c818-41c2-86a6-11f297288a35-istio-podinfo\") pod \"cff875c0-c818-41c2-86a6-11f297288a35\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") "
Sep 08 15:36:17.160206 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160162    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-istio-envoy\") pod \"cff875c0-c818-41c2-86a6-11f297288a35\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") "
Sep 08 15:36:17.160206 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160202    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-workload-certs\") pod \"cff875c0-c818-41c2-86a6-11f297288a35\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") "
Sep 08 15:36:17.160417 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160300    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-workload-socket\") pod \"cff875c0-c818-41c2-86a6-11f297288a35\" (UID: \"cff875c0-c818-41c2-86a6-11f297288a35\") "
Sep 08 15:36:17.160417 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160369    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-istio-data" (OuterVolumeSpecName: "istio-data") pod "cff875c0-c818-41c2-86a6-11f297288a35" (UID: "cff875c0-c818-41c2-86a6-11f297288a35"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:36:17.160417 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160401    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cff875c0-c818-41c2-86a6-11f297288a35-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cff875c0-c818-41c2-86a6-11f297288a35" (UID: "cff875c0-c818-41c2-86a6-11f297288a35"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:36:17.160654 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160621    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cff875c0-c818-41c2-86a6-11f297288a35" (UID: "cff875c0-c818-41c2-86a6-11f297288a35"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:36:17.160729 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160667    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cff875c0-c818-41c2-86a6-11f297288a35" (UID: "cff875c0-c818-41c2-86a6-11f297288a35"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:36:17.160785 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160732    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cff875c0-c818-41c2-86a6-11f297288a35" (UID: "cff875c0-c818-41c2-86a6-11f297288a35"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:36:17.160785 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160745    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cff875c0-c818-41c2-86a6-11f297288a35-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:17.160785 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160763    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:17.160785 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.160776    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:17.162750 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.162726    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cff875c0-c818-41c2-86a6-11f297288a35-istio-token" (OuterVolumeSpecName: "istio-token") pod "cff875c0-c818-41c2-86a6-11f297288a35" (UID: "cff875c0-c818-41c2-86a6-11f297288a35"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:36:17.163134 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.163113    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cff875c0-c818-41c2-86a6-11f297288a35-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cff875c0-c818-41c2-86a6-11f297288a35" (UID: "cff875c0-c818-41c2-86a6-11f297288a35"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:36:17.163205 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.163141    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cff875c0-c818-41c2-86a6-11f297288a35" (UID: "cff875c0-c818-41c2-86a6-11f297288a35"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:36:17.163272 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.163253    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cff875c0-c818-41c2-86a6-11f297288a35-kube-api-access-tqnbr" (OuterVolumeSpecName: "kube-api-access-tqnbr") pod "cff875c0-c818-41c2-86a6-11f297288a35" (UID: "cff875c0-c818-41c2-86a6-11f297288a35"). InnerVolumeSpecName "kube-api-access-tqnbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:36:17.182497 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.182460    2575 scope.go:117] "RemoveContainer" containerID="51fee8dfc195e41de149ce80b861015fa873f70855e592bb3f9c67a86bd00280"
Sep 08 15:36:17.182880 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:36:17.182859    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51fee8dfc195e41de149ce80b861015fa873f70855e592bb3f9c67a86bd00280\": container with ID starting with 51fee8dfc195e41de149ce80b861015fa873f70855e592bb3f9c67a86bd00280 not found: ID does not exist" containerID="51fee8dfc195e41de149ce80b861015fa873f70855e592bb3f9c67a86bd00280"
Sep 08 15:36:17.182934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.182889    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51fee8dfc195e41de149ce80b861015fa873f70855e592bb3f9c67a86bd00280"} err="failed to get container status \"51fee8dfc195e41de149ce80b861015fa873f70855e592bb3f9c67a86bd00280\": rpc error: code = NotFound desc = could not find container \"51fee8dfc195e41de149ce80b861015fa873f70855e592bb3f9c67a86bd00280\": container with ID starting with 51fee8dfc195e41de149ce80b861015fa873f70855e592bb3f9c67a86bd00280 not found: ID does not exist"
Sep 08 15:36:17.182934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.182907    2575 scope.go:117] "RemoveContainer" containerID="d7e637326aa7b35c8baaeef59fbe7576a0ffc3982c9ff1d6fcbdce015d4044c1"
Sep 08 15:36:17.183202 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:36:17.183180    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7e637326aa7b35c8baaeef59fbe7576a0ffc3982c9ff1d6fcbdce015d4044c1\": container with ID starting with d7e637326aa7b35c8baaeef59fbe7576a0ffc3982c9ff1d6fcbdce015d4044c1 not found: ID does not exist" containerID="d7e637326aa7b35c8baaeef59fbe7576a0ffc3982c9ff1d6fcbdce015d4044c1"
Sep 08 15:36:17.183249 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.183211    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7e637326aa7b35c8baaeef59fbe7576a0ffc3982c9ff1d6fcbdce015d4044c1"} err="failed to get container status \"d7e637326aa7b35c8baaeef59fbe7576a0ffc3982c9ff1d6fcbdce015d4044c1\": rpc error: code = NotFound desc = could not find container \"d7e637326aa7b35c8baaeef59fbe7576a0ffc3982c9ff1d6fcbdce015d4044c1\": container with ID starting with d7e637326aa7b35c8baaeef59fbe7576a0ffc3982c9ff1d6fcbdce015d4044c1 not found: ID does not exist"
Sep 08 15:36:17.261686 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.261652    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqnbr\" (UniqueName: \"kubernetes.io/projected/cff875c0-c818-41c2-86a6-11f297288a35-kube-api-access-tqnbr\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:17.261686 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.261680    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cff875c0-c818-41c2-86a6-11f297288a35-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:17.261686 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.261690    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:17.261923 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.261699    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:17.261923 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.261707    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cff875c0-c818-41c2-86a6-11f297288a35-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:17.261923 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.261714    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cff875c0-c818-41c2-86a6-11f297288a35-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:36:17.475480 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.475444    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"]
Sep 08 15:36:17.479692 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.479664    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ae34c54a-openshift-default-6b9b8bd4bd-r7tdp"]
Sep 08 15:36:17.843242 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:17.843146    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:36:18.433351 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:18.433318    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cff875c0-c818-41c2-86a6-11f297288a35" path="/var/lib/kubelet/pods/cff875c0-c818-41c2-86a6-11f297288a35/volumes"
Sep 08 15:36:21.546341 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:21.546307    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:21.643562 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:21.643530    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:21.758295 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:21.758258    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:22.263693 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:22.263658    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:22.363835 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:22.363796    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:22.955366 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:22.955329    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:23.267259 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:23.267168    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:28.925412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.925376    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"]
Sep 08 15:36:28.925891 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.925863    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cff875c0-c818-41c2-86a6-11f297288a35" containerName="istio-proxy"
Sep 08 15:36:28.925891 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.925877    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cff875c0-c818-41c2-86a6-11f297288a35" containerName="istio-proxy"
Sep 08 15:36:28.925891 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.925889    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cff875c0-c818-41c2-86a6-11f297288a35" containerName="istio-proxy"
Sep 08 15:36:28.926016 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.925896    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cff875c0-c818-41c2-86a6-11f297288a35" containerName="istio-proxy"
Sep 08 15:36:28.926016 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.925902    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cab2d97-430d-4920-b4b7-cfcf641e39dd" containerName="main"
Sep 08 15:36:28.926016 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.925909    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cab2d97-430d-4920-b4b7-cfcf641e39dd" containerName="main"
Sep 08 15:36:28.926016 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.925973    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cff875c0-c818-41c2-86a6-11f297288a35" containerName="istio-proxy"
Sep 08 15:36:28.926016 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.925983    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cff875c0-c818-41c2-86a6-11f297288a35" containerName="istio-proxy"
Sep 08 15:36:28.926016 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.925997    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cab2d97-430d-4920-b4b7-cfcf641e39dd" containerName="main"
Sep 08 15:36:28.930665 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.930642    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:28.934549 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.934523    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-bc9cdf\"/\"rate-test-model-bc9cdf-kserve-self-signed-certs\""
Sep 08 15:36:28.934695 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.934523    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-bc9cdf\"/\"default-dockercfg-xpclp\""
Sep 08 15:36:28.934695 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.934524    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-bc9cdf\"/\"openshift-service-ca.crt\""
Sep 08 15:36:28.934695 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.934524    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-bc9cdf\"/\"kube-root-ca.crt\""
Sep 08 15:36:28.942607 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.942561    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"]
Sep 08 15:36:28.958460 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.958423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:28.974570 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.974531    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-tls-certs\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:28.974570 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.974571    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv9bm\" (UniqueName: \"kubernetes.io/projected/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-kube-api-access-tv9bm\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:28.974789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.974673    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-dshm\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:28.974789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.974736    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-home\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:28.974789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.974757    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-model-cache\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:28.974894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:28.974803    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-tmp-dir\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.076244 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.076185    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-dshm\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.076561 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.076267    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-home\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.076561 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.076288    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-model-cache\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.076561 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.076319    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-tmp-dir\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.076561 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.076452    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-tls-certs\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.076561 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.076514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv9bm\" (UniqueName: \"kubernetes.io/projected/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-kube-api-access-tv9bm\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.076862 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.076774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-tmp-dir\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.076862 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.076803    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-model-cache\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.076862 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.076832    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-home\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.078615 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.078592    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-dshm\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.078892 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.078875    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-tls-certs\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.088080 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.088054    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv9bm\" (UniqueName: \"kubernetes.io/projected/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-kube-api-access-tv9bm\") pod \"rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") " pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.238533 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.238428    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:29.244273 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.244242    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:29.361124 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.361087    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:29.462993 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.462951    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:29.524910 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:29.524884    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"]
Sep 08 15:36:29.527059 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:36:29.527027    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59b6c7cc_3500_4d00_82eb_9ceb9fa82453.slice/crio-3640e75ad5e08895b832baa5a66aa8fb8cf56eb0d9d3ddbbfe23c512ee289a63 WatchSource:0}: Error finding container 3640e75ad5e08895b832baa5a66aa8fb8cf56eb0d9d3ddbbfe23c512ee289a63: Status 404 returned error can't find the container with id 3640e75ad5e08895b832baa5a66aa8fb8cf56eb0d9d3ddbbfe23c512ee289a63
Sep 08 15:36:30.225573 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:30.225535    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm" event={"ID":"59b6c7cc-3500-4d00-82eb-9ceb9fa82453","Type":"ContainerStarted","Data":"efa94e0af559b99071cdb8a0b275d5fa2ecffc634b66910244d6bb3f7133b33a"}
Sep 08 15:36:30.225573 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:30.225579    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm" event={"ID":"59b6c7cc-3500-4d00-82eb-9ceb9fa82453","Type":"ContainerStarted","Data":"3640e75ad5e08895b832baa5a66aa8fb8cf56eb0d9d3ddbbfe23c512ee289a63"}
Sep 08 15:36:30.250124 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:30.250053    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm" podStartSLOduration=2.250037955 podStartE2EDuration="2.250037955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:36:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:36:30.248809312 +0000 UTC m=+1638.334759354" watchObservedRunningTime="2026-09-08 15:36:30.250037955 +0000 UTC m=+1638.335987996"
Sep 08 15:36:32.760389 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:32.760350    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:36:39.244626 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:39.244571    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:39.245002 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:39.244651    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:39.257113 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:39.257087    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:39.282798 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:39.282774    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:36:40.024672 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:40.024628    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:40.315913 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:40.315822    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:36:48.963154 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:48.963112    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:36:53.303480 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:36:53.303440    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:01.251009 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:01.250960    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:07.352271 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:07.352233    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:37:07.708295 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:07.708262    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:07.799546 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:07.799504    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:08.162546 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.162499    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:08.460777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.460687    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"]
Sep 08 15:37:08.464593 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.464573    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.467508 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.467471    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-61a1d6-openshift-default-dockercfg-2gcxn\""
Sep 08 15:37:08.479443 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.479416    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"]
Sep 08 15:37:08.553042 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.553010    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:08.558575 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.558543    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-data\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.558753 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.558598    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-podinfo\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.558832 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.558698    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-workload-certs\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.558891 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.558867    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tb9g\" (UniqueName: \"kubernetes.io/projected/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-kube-api-access-5tb9g\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.558952 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.558903    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-credential-socket\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.558952 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.558927    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-token\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.559059 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.558997    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-workload-socket\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.559059 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.559046    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-envoy\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.559167 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.559100    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istiod-ca-cert\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.660148 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660111    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-workload-socket\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.660330 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-envoy\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.660330 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660195    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istiod-ca-cert\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.660330 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660264    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-data\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.660330 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660305    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-podinfo\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.660589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660356    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-workload-certs\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.660589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660423    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tb9g\" (UniqueName: \"kubernetes.io/projected/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-kube-api-access-5tb9g\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.660589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660456    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-credential-socket\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.660589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660482    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-token\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.660589 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-workload-socket\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.660944 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660913    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-workload-certs\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.661049 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660950    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-data\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.661049 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660979    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-credential-socket\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.661049 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.660983    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istiod-ca-cert\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.662536 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.662515    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-envoy\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.662742 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.662722    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-podinfo\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.668900 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.668874    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-token\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.669422 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.669404    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tb9g\" (UniqueName: \"kubernetes.io/projected/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-kube-api-access-5tb9g\") pod \"e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") " pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:08.766730 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.766664    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:08.774643 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:08.774621    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:09.056369 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:09.056346    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"]
Sep 08 15:37:09.057449 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:37:09.057428    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ff6cd8a_5ffc_4392_a9be_544a15678ed1.slice/crio-a25a23cf091c937c0e177199a9d0c8f48e31f6bb5cf37499771e90f636c4bf26 WatchSource:0}: Error finding container a25a23cf091c937c0e177199a9d0c8f48e31f6bb5cf37499771e90f636c4bf26: Status 404 returned error can't find the container with id a25a23cf091c937c0e177199a9d0c8f48e31f6bb5cf37499771e90f636c4bf26
Sep 08 15:37:09.059802 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:09.059762    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:09.059894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:09.059869    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:09.059935 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:09.059921    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:09.433428 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:09.433388    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" event={"ID":"2ff6cd8a-5ffc-4392-a9be-544a15678ed1","Type":"ContainerStarted","Data":"e037ea345efdbfbc248d2d2ea556c0b8a3ca58acf4676285fef2b490c07116b8"}
Sep 08 15:37:09.433610 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:09.433443    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" event={"ID":"2ff6cd8a-5ffc-4392-a9be-544a15678ed1","Type":"ContainerStarted","Data":"a25a23cf091c937c0e177199a9d0c8f48e31f6bb5cf37499771e90f636c4bf26"}
Sep 08 15:37:09.456849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:09.456804    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" podStartSLOduration=1.4567908059999999 podStartE2EDuration="1.456790806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:37:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:37:09.45483812 +0000 UTC m=+1677.540788184" watchObservedRunningTime="2026-09-08 15:37:09.456790806 +0000 UTC m=+1677.542740847"
Sep 08 15:37:09.775093 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:09.775023    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:10.780973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:10.780944    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:11.444300 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:11.444266    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:11.445190 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:11.445170    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:11.489360 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:11.489326    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:11.587969 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:11.587936    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:12.034552 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.034520    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"]
Sep 08 15:37:12.045376 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.045345    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:12.599524 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.599469    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"]
Sep 08 15:37:12.605472 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.605431    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:12.608587 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.608563    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-61a1d6-serving-cert\""
Sep 08 15:37:12.618700 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.618667    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"]
Sep 08 15:37:12.650697 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.650670    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:12.701864 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.701838    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-metrics-certs\") pod \"maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:12.701966 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.701882    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7010f537-f8dd-4aef-8c86-551ecd79c7e0-openshift-service-ca\") pod \"maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:12.701966 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.701904    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-maas-api-tls\") pod \"maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:12.702089 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.702012    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7p7b\" (UniqueName: \"kubernetes.io/projected/7010f537-f8dd-4aef-8c86-551ecd79c7e0-kube-api-access-f7p7b\") pod \"maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:12.739008 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.738970    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:12.803552 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.803522    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-metrics-certs\") pod \"maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:12.803653 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.803573    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7010f537-f8dd-4aef-8c86-551ecd79c7e0-openshift-service-ca\") pod \"maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:12.803653 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.803608    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-maas-api-tls\") pod \"maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:12.803743 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.803652    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7p7b\" (UniqueName: \"kubernetes.io/projected/7010f537-f8dd-4aef-8c86-551ecd79c7e0-kube-api-access-f7p7b\") pod \"maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:12.803743 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:37:12.803734    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-serving-cert: secret "maas-api-e2e-api-a-61a1d6-serving-cert" not found
Sep 08 15:37:12.803821 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:37:12.803809    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-maas-api-tls podName:7010f537-f8dd-4aef-8c86-551ecd79c7e0 nodeName:}" failed. No retries permitted until 2026-09-08 15:37:13.303786934 +0000 UTC m=+1681.389736955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-maas-api-tls") pod "maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8" (UID: "7010f537-f8dd-4aef-8c86-551ecd79c7e0") : secret "maas-api-e2e-api-a-61a1d6-serving-cert" not found
Sep 08 15:37:12.804319 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.804302    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7010f537-f8dd-4aef-8c86-551ecd79c7e0-openshift-service-ca\") pod \"maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:12.806014 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.805989    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-metrics-certs\") pod \"maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:12.812998 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.812978    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7p7b\" (UniqueName: \"kubernetes.io/projected/7010f537-f8dd-4aef-8c86-551ecd79c7e0-kube-api-access-f7p7b\") pod \"maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:12.995970 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:12.995939    2575 scope.go:117] "RemoveContainer" containerID="d126dd518f5aa5cef218f1a4a1069b19dae4fa60d99dc9905eeb98d922efb592"
Sep 08 15:37:13.083616 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:13.083587    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:13.185807 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:13.185772    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:13.280980 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:13.280899    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:13.307506 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:13.307464    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-maas-api-tls\") pod \"maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:13.310020 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:13.309998    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-maas-api-tls\") pod \"maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:13.518880 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:13.518853    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:13.796243 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:13.796219    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"]
Sep 08 15:37:13.798000 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:37:13.797974    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7010f537_f8dd_4aef_8c86_551ecd79c7e0.slice/crio-e748ad70dc5c149c82fb9192e950972408a7883e8fdeda7a89820d5d0d2d4576 WatchSource:0}: Error finding container e748ad70dc5c149c82fb9192e950972408a7883e8fdeda7a89820d5d0d2d4576: Status 404 returned error can't find the container with id e748ad70dc5c149c82fb9192e950972408a7883e8fdeda7a89820d5d0d2d4576
Sep 08 15:37:13.799469 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:13.799452    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:37:13.934904 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:13.934873    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"]
Sep 08 15:37:13.957138 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:13.957086    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:14.448853 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.448823    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:14.461136 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.461089    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8" event={"ID":"7010f537-f8dd-4aef-8c86-551ecd79c7e0","Type":"ContainerStarted","Data":"eb5628e89cc6a480f48220ac555a68c0bb8a77a06e276eb17fc596d040f3980c"}
Sep 08 15:37:14.461337 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.461152    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8" event={"ID":"7010f537-f8dd-4aef-8c86-551ecd79c7e0","Type":"ContainerStarted","Data":"e748ad70dc5c149c82fb9192e950972408a7883e8fdeda7a89820d5d0d2d4576"}
Sep 08 15:37:14.461337 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.461208    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:14.484316 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.484266    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8" podStartSLOduration=1.9042108290000002 podStartE2EDuration="2.48425181s" totalImagesPullingTime="580.040981ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:37:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:37:14.483044019 +0000 UTC m=+1682.568994072" watchObservedRunningTime="2026-09-08 15:37:14.48425181 +0000 UTC m=+1682.570201852"
Sep 08 15:37:14.730892 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.730809    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"]
Sep 08 15:37:14.734300 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.734279    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.737272 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.737241    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-4a2c80-openshift-default-dockercfg-kpsts\""
Sep 08 15:37:14.749813 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.749790    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"]
Sep 08 15:37:14.771587 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.771550    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:14.823838 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.823787    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-workload-certs\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.824012 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.823920    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-token\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.824012 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.823955    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istiod-ca-cert\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.824012 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.824002    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-workload-socket\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.824123 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.824032    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-credential-socket\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.824123 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.824054    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-data\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.824188 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.824113    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-envoy\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.824188 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.824149    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dldw8\" (UniqueName: \"kubernetes.io/projected/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-kube-api-access-dldw8\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.824188 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.824176    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-podinfo\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.925126 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.925089    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-envoy\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.925275 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.925137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dldw8\" (UniqueName: \"kubernetes.io/projected/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-kube-api-access-dldw8\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.925275 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.925173    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-podinfo\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.925275 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.925237    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-workload-certs\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.925417 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.925327    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-token\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.925417 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.925357    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istiod-ca-cert\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.925417 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.925409    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-workload-socket\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.925603 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.925446    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-credential-socket\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.925603 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.925481    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-data\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.925730 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.925696    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-workload-certs\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.925862 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.925731    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-credential-socket\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.925862 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.925800    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-workload-socket\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.925981 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.925959    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-data\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.926237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.926218    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istiod-ca-cert\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.927419 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.927400    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-envoy\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.927549 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.927532    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-podinfo\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.934178 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.934157    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-token\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:14.934315 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:14.934297    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dldw8\" (UniqueName: \"kubernetes.io/projected/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-kube-api-access-dldw8\") pod \"e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") " pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:15.045696 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:15.045614    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:15.328968 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:15.328936    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"]
Sep 08 15:37:15.330336 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:15.330305    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:15.331199 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:37:15.331168    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34bfc4d_2e1d_4a5d_a2d5_68e05c5b31d6.slice/crio-7d7ed6ea917a4f1cd67e7f86df615dbc1a4c20ed23fae1e67cb6941014c885e4 WatchSource:0}: Error finding container 7d7ed6ea917a4f1cd67e7f86df615dbc1a4c20ed23fae1e67cb6941014c885e4: Status 404 returned error can't find the container with id 7d7ed6ea917a4f1cd67e7f86df615dbc1a4c20ed23fae1e67cb6941014c885e4
Sep 08 15:37:15.333601 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:15.333571    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:15.333681 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:15.333634    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:15.333681 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:15.333663    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:15.468431 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:15.468389    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" event={"ID":"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6","Type":"ContainerStarted","Data":"a359a057666ff7c0814a65b10713f37f11896b3d384aeaac26c679d17ceb9fe4"}
Sep 08 15:37:15.468811 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:15.468461    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" event={"ID":"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6","Type":"ContainerStarted","Data":"7d7ed6ea917a4f1cd67e7f86df615dbc1a4c20ed23fae1e67cb6941014c885e4"}
Sep 08 15:37:15.493638 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:15.493583    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" podStartSLOduration=1.4935666300000001 podStartE2EDuration="1.49356663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:37:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:37:15.491023472 +0000 UTC m=+1683.576973527" watchObservedRunningTime="2026-09-08 15:37:15.49356663 +0000 UTC m=+1683.579516671"
Sep 08 15:37:16.046255 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:16.046223    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:16.051072 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:16.051050    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:16.476340 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:16.476308    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:16.477264 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:16.477243    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:17.479350 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:17.479311    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:17.607660 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:17.607627    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:18.619618 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:18.619587    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:18.717507 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:18.717464    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:18.821515 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:18.821458    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:19.118287 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:19.118252    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:19.227336 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:19.227301    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:20.474131 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:20.474104    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:37:24.138752 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:24.138716    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:37:27.054228 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.054161    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl" podUID="a4eafea6-a344-4962-80d5-59000507b236" containerName="main" containerID="cri-o://d42f47329b0f2eaaaf3b348196038d85d50609e8977278df9260ad30d813f74a" gracePeriod=45
Sep 08 15:37:27.446649 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.446618    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:37:27.548701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.548184    2575 generic.go:356] "Generic (PLEG): container finished" podID="a4eafea6-a344-4962-80d5-59000507b236" containerID="d42f47329b0f2eaaaf3b348196038d85d50609e8977278df9260ad30d813f74a" exitCode=0
Sep 08 15:37:27.548701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.548241    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl" event={"ID":"a4eafea6-a344-4962-80d5-59000507b236","Type":"ContainerDied","Data":"d42f47329b0f2eaaaf3b348196038d85d50609e8977278df9260ad30d813f74a"}
Sep 08 15:37:27.548701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.548268    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl" event={"ID":"a4eafea6-a344-4962-80d5-59000507b236","Type":"ContainerDied","Data":"83e618936326c1dc7ebc5567f4754bd0f584227e89b7b13c2458e041dabc79f6"}
Sep 08 15:37:27.548701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.548284    2575 scope.go:117] "RemoveContainer" containerID="d42f47329b0f2eaaaf3b348196038d85d50609e8977278df9260ad30d813f74a"
Sep 08 15:37:27.548701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.548405    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"
Sep 08 15:37:27.556144 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.556111    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-model-cache\") pod \"a4eafea6-a344-4962-80d5-59000507b236\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") "
Sep 08 15:37:27.556285 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.556171    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-tmp-dir\") pod \"a4eafea6-a344-4962-80d5-59000507b236\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") "
Sep 08 15:37:27.556285 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.556217    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4eafea6-a344-4962-80d5-59000507b236-tls-certs\") pod \"a4eafea6-a344-4962-80d5-59000507b236\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") "
Sep 08 15:37:27.556358 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.556311    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-home\") pod \"a4eafea6-a344-4962-80d5-59000507b236\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") "
Sep 08 15:37:27.556358 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.556343    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gw9ht\" (UniqueName: \"kubernetes.io/projected/a4eafea6-a344-4962-80d5-59000507b236-kube-api-access-gw9ht\") pod \"a4eafea6-a344-4962-80d5-59000507b236\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") "
Sep 08 15:37:27.556427 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.556415    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-dshm\") pod \"a4eafea6-a344-4962-80d5-59000507b236\" (UID: \"a4eafea6-a344-4962-80d5-59000507b236\") "
Sep 08 15:37:27.556905 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.556743    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4eafea6-a344-4962-80d5-59000507b236" (UID: "a4eafea6-a344-4962-80d5-59000507b236"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:37:27.556905 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.556752    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4eafea6-a344-4962-80d5-59000507b236" (UID: "a4eafea6-a344-4962-80d5-59000507b236"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:37:27.557108 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.556940    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-home" (OuterVolumeSpecName: "home") pod "a4eafea6-a344-4962-80d5-59000507b236" (UID: "a4eafea6-a344-4962-80d5-59000507b236"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:37:27.557108 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.557099    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:37:27.557202 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.557118    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:37:27.557202 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.557131    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:37:27.557921 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.557904    2575 scope.go:117] "RemoveContainer" containerID="d42f47329b0f2eaaaf3b348196038d85d50609e8977278df9260ad30d813f74a"
Sep 08 15:37:27.558245 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:37:27.558190    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d42f47329b0f2eaaaf3b348196038d85d50609e8977278df9260ad30d813f74a\": container with ID starting with d42f47329b0f2eaaaf3b348196038d85d50609e8977278df9260ad30d813f74a not found: ID does not exist" containerID="d42f47329b0f2eaaaf3b348196038d85d50609e8977278df9260ad30d813f74a"
Sep 08 15:37:27.558245 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.558218    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d42f47329b0f2eaaaf3b348196038d85d50609e8977278df9260ad30d813f74a"} err="failed to get container status \"d42f47329b0f2eaaaf3b348196038d85d50609e8977278df9260ad30d813f74a\": rpc error: code = NotFound desc = could not find container \"d42f47329b0f2eaaaf3b348196038d85d50609e8977278df9260ad30d813f74a\": container with ID starting with d42f47329b0f2eaaaf3b348196038d85d50609e8977278df9260ad30d813f74a not found: ID does not exist"
Sep 08 15:37:27.559120 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.559098    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-dshm" (OuterVolumeSpecName: "dshm") pod "a4eafea6-a344-4962-80d5-59000507b236" (UID: "a4eafea6-a344-4962-80d5-59000507b236"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:37:27.559309 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.559290    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4eafea6-a344-4962-80d5-59000507b236-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4eafea6-a344-4962-80d5-59000507b236" (UID: "a4eafea6-a344-4962-80d5-59000507b236"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:37:27.559551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.559530    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4eafea6-a344-4962-80d5-59000507b236-kube-api-access-gw9ht" (OuterVolumeSpecName: "kube-api-access-gw9ht") pod "a4eafea6-a344-4962-80d5-59000507b236" (UID: "a4eafea6-a344-4962-80d5-59000507b236"). InnerVolumeSpecName "kube-api-access-gw9ht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:37:27.658280 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.658184    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gw9ht\" (UniqueName: \"kubernetes.io/projected/a4eafea6-a344-4962-80d5-59000507b236-kube-api-access-gw9ht\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:37:27.658280 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.658221    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4eafea6-a344-4962-80d5-59000507b236-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:37:27.658280 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.658235    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4eafea6-a344-4962-80d5-59000507b236-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:37:27.872321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.872287    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"]
Sep 08 15:37:27.875963 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:27.875940    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2e082e/rate-test-model-2e082e-kserve-76bc465b95-b29fl"]
Sep 08 15:37:28.441537 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:28.441483    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4eafea6-a344-4962-80d5-59000507b236" path="/var/lib/kubelet/pods/a4eafea6-a344-4962-80d5-59000507b236/volumes"
Sep 08 15:37:28.954083 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:28.954043    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm" podUID="59b6c7cc-3500-4d00-82eb-9ceb9fa82453" containerName="main" containerID="cri-o://efa94e0af559b99071cdb8a0b275d5fa2ecffc634b66910244d6bb3f7133b33a" gracePeriod=45
Sep 08 15:37:29.287472 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.287450    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:37:29.478237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.477558    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-model-cache\") pod \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") "
Sep 08 15:37:29.478237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.477596    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tv9bm\" (UniqueName: \"kubernetes.io/projected/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-kube-api-access-tv9bm\") pod \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") "
Sep 08 15:37:29.478237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.477652    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-tmp-dir\") pod \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") "
Sep 08 15:37:29.478237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.477706    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-tls-certs\") pod \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") "
Sep 08 15:37:29.478237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.477730    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-dshm\") pod \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") "
Sep 08 15:37:29.478237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.477781    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-home\") pod \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\" (UID: \"59b6c7cc-3500-4d00-82eb-9ceb9fa82453\") "
Sep 08 15:37:29.478237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.477790    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-model-cache" (OuterVolumeSpecName: "model-cache") pod "59b6c7cc-3500-4d00-82eb-9ceb9fa82453" (UID: "59b6c7cc-3500-4d00-82eb-9ceb9fa82453"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:37:29.478237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.477904    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59b6c7cc-3500-4d00-82eb-9ceb9fa82453" (UID: "59b6c7cc-3500-4d00-82eb-9ceb9fa82453"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:37:29.478237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.477992    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:37:29.478237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.478002    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:37:29.478237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.478048    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-home" (OuterVolumeSpecName: "home") pod "59b6c7cc-3500-4d00-82eb-9ceb9fa82453" (UID: "59b6c7cc-3500-4d00-82eb-9ceb9fa82453"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:37:29.480020 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.479993    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-kube-api-access-tv9bm" (OuterVolumeSpecName: "kube-api-access-tv9bm") pod "59b6c7cc-3500-4d00-82eb-9ceb9fa82453" (UID: "59b6c7cc-3500-4d00-82eb-9ceb9fa82453"). InnerVolumeSpecName "kube-api-access-tv9bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:37:29.480128 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.480036    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59b6c7cc-3500-4d00-82eb-9ceb9fa82453" (UID: "59b6c7cc-3500-4d00-82eb-9ceb9fa82453"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:37:29.480128 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.480094    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-dshm" (OuterVolumeSpecName: "dshm") pod "59b6c7cc-3500-4d00-82eb-9ceb9fa82453" (UID: "59b6c7cc-3500-4d00-82eb-9ceb9fa82453"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:37:29.560895 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.560808    2575 generic.go:356] "Generic (PLEG): container finished" podID="59b6c7cc-3500-4d00-82eb-9ceb9fa82453" containerID="efa94e0af559b99071cdb8a0b275d5fa2ecffc634b66910244d6bb3f7133b33a" exitCode=0
Sep 08 15:37:29.560895 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.560851    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"
Sep 08 15:37:29.561074 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.560946    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm" event={"ID":"59b6c7cc-3500-4d00-82eb-9ceb9fa82453","Type":"ContainerDied","Data":"efa94e0af559b99071cdb8a0b275d5fa2ecffc634b66910244d6bb3f7133b33a"}
Sep 08 15:37:29.561074 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.560973    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm" event={"ID":"59b6c7cc-3500-4d00-82eb-9ceb9fa82453","Type":"ContainerDied","Data":"3640e75ad5e08895b832baa5a66aa8fb8cf56eb0d9d3ddbbfe23c512ee289a63"}
Sep 08 15:37:29.561074 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.560988    2575 scope.go:117] "RemoveContainer" containerID="efa94e0af559b99071cdb8a0b275d5fa2ecffc634b66910244d6bb3f7133b33a"
Sep 08 15:37:29.570196 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.570157    2575 scope.go:117] "RemoveContainer" containerID="efa94e0af559b99071cdb8a0b275d5fa2ecffc634b66910244d6bb3f7133b33a"
Sep 08 15:37:29.570475 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:37:29.570430    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efa94e0af559b99071cdb8a0b275d5fa2ecffc634b66910244d6bb3f7133b33a\": container with ID starting with efa94e0af559b99071cdb8a0b275d5fa2ecffc634b66910244d6bb3f7133b33a not found: ID does not exist" containerID="efa94e0af559b99071cdb8a0b275d5fa2ecffc634b66910244d6bb3f7133b33a"
Sep 08 15:37:29.570475 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.570457    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efa94e0af559b99071cdb8a0b275d5fa2ecffc634b66910244d6bb3f7133b33a"} err="failed to get container status \"efa94e0af559b99071cdb8a0b275d5fa2ecffc634b66910244d6bb3f7133b33a\": rpc error: code = NotFound desc = could not find container \"efa94e0af559b99071cdb8a0b275d5fa2ecffc634b66910244d6bb3f7133b33a\": container with ID starting with efa94e0af559b99071cdb8a0b275d5fa2ecffc634b66910244d6bb3f7133b33a not found: ID does not exist"
Sep 08 15:37:29.579284 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.579265    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:37:29.579284 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.579283    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:37:29.579397 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.579292    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:37:29.579397 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.579300    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv9bm\" (UniqueName: \"kubernetes.io/projected/59b6c7cc-3500-4d00-82eb-9ceb9fa82453-kube-api-access-tv9bm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:37:29.582973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.582954    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"]
Sep 08 15:37:29.585000 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.584978    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm"]
Sep 08 15:37:29.585104 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.585078    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:29.686824 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:29.686794    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:30.438032 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:30.437478    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59b6c7cc-3500-4d00-82eb-9ceb9fa82453" path="/var/lib/kubelet/pods/59b6c7cc-3500-4d00-82eb-9ceb9fa82453/volumes"
Sep 08 15:37:31.703878 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:31.703787    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:32.518959 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:32.518922    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 15:37:32.519210 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:32.519177    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 15:37:32.928799 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:32.928721    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:33.723601 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:33.723555    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:34.271042 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:34.270983    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w3-bc9cdf/rate-test-model-bc9cdf-kserve-96b9df68c-fnjmm" podUID="59b6c7cc-3500-4d00-82eb-9ceb9fa82453" containerName="main" probeResult="failure" output="Get \"https://10.134.0.76:8000/ready\": context deadline exceeded"
Sep 08 15:37:35.599646 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:35.599614    2575 generic.go:356] "Generic (PLEG): container finished" podID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerID="a359a057666ff7c0814a65b10713f37f11896b3d384aeaac26c679d17ceb9fe4" exitCode=0
Sep 08 15:37:35.600132 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:35.599687    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" event={"ID":"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6","Type":"ContainerDied","Data":"a359a057666ff7c0814a65b10713f37f11896b3d384aeaac26c679d17ceb9fe4"}
Sep 08 15:37:35.600237 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:35.600217    2575 scope.go:117] "RemoveContainer" containerID="a359a057666ff7c0814a65b10713f37f11896b3d384aeaac26c679d17ceb9fe4"
Sep 08 15:37:35.601569 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:35.601546    2575 generic.go:356] "Generic (PLEG): container finished" podID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerID="e037ea345efdbfbc248d2d2ea556c0b8a3ca58acf4676285fef2b490c07116b8" exitCode=0
Sep 08 15:37:35.601664 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:35.601591    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" event={"ID":"2ff6cd8a-5ffc-4392-a9be-544a15678ed1","Type":"ContainerDied","Data":"e037ea345efdbfbc248d2d2ea556c0b8a3ca58acf4676285fef2b490c07116b8"}
Sep 08 15:37:35.601664 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:35.601616    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:35.601737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:35.601695    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:35.601737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:35.601732    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:35.601932 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:35.601919    2575 scope.go:117] "RemoveContainer" containerID="e037ea345efdbfbc248d2d2ea556c0b8a3ca58acf4676285fef2b490c07116b8"
Sep 08 15:37:35.602974 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:35.602946    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:35.603062 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:35.603008    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:35.603062 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:35.603042    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:36.479074 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:36.479033    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:36.614774 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:36.614730    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" event={"ID":"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6","Type":"ContainerStarted","Data":"cabbbc573c0a4cc01a9cdea421ed70be8e9d21f0cb9cf0addcdd564770b13c8f"}
Sep 08 15:37:36.618101 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:36.618068    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" event={"ID":"2ff6cd8a-5ffc-4392-a9be-544a15678ed1","Type":"ContainerStarted","Data":"570d58dc5c8e82eb72f868f0894da77853b6ead596bdf6af11458075892e7004"}
Sep 08 15:37:36.774922 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:36.774814    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:36.776450 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:36.776413    2575 patch_prober.go:28] interesting pod/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 08 15:37:36.776617 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:36.776504    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 08 15:37:36.951644 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:36.951598    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:37.046652 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.046528    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:37.046871 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.046734    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 08 15:37:37.046871 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.046782    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 08 15:37:37.289058 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.289026    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"]
Sep 08 15:37:37.308428 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.289521    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4eafea6-a344-4962-80d5-59000507b236" containerName="main"
Sep 08 15:37:37.308428 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.289542    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4eafea6-a344-4962-80d5-59000507b236" containerName="main"
Sep 08 15:37:37.308428 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.289576    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59b6c7cc-3500-4d00-82eb-9ceb9fa82453" containerName="main"
Sep 08 15:37:37.308428 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.289585    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="59b6c7cc-3500-4d00-82eb-9ceb9fa82453" containerName="main"
Sep 08 15:37:37.308428 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.289718    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="59b6c7cc-3500-4d00-82eb-9ceb9fa82453" containerName="main"
Sep 08 15:37:37.308428 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.289731    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4eafea6-a344-4962-80d5-59000507b236" containerName="main"
Sep 08 15:37:37.308428 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.292406    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.308428 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.297162    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ee0da5-openshift-default-dockercfg-djjn9\""
Sep 08 15:37:37.322896 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.317311    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"]
Sep 08 15:37:37.357732 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.357687    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:37.375647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.374559    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-envoy\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.375647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.374632    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-workload-socket\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.375647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.374689    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istiod-ca-cert\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.375647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.374790    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-workload-certs\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.375647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.374898    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-data\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.375647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.374956    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-credential-socket\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.375647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.375078    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-podinfo\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.375647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.375105    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86r4g\" (UniqueName: \"kubernetes.io/projected/4b28902e-c61b-437e-92bb-94e8bb7f9d94-kube-api-access-86r4g\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.375647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.375133    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-token\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.476070 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.476018    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-workload-socket\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.476070 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.476076    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istiod-ca-cert\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.476303 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.476136    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-workload-certs\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.476303 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.476207    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-data\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.476303 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.476230    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-credential-socket\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.476303 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.476294    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-podinfo\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.476445 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.476312    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86r4g\" (UniqueName: \"kubernetes.io/projected/4b28902e-c61b-437e-92bb-94e8bb7f9d94-kube-api-access-86r4g\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.476445 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.476331    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-token\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.476445 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.476377    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-envoy\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.476445 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.476411    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-workload-socket\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.476934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.476704    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-data\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.476934 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.476805    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istiod-ca-cert\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.477125 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.477041    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-workload-certs\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.477187 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.477136    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-credential-socket\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.478948 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.478924    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-envoy\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.479287 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.479270    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-podinfo\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.488350 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.488318    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86r4g\" (UniqueName: \"kubernetes.io/projected/4b28902e-c61b-437e-92bb-94e8bb7f9d94-kube-api-access-86r4g\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.488713 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.488693    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-token\") pod \"e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") " pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.627788 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.627750    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:37.669890 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.669800    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:37.776104 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.775962    2575 patch_prober.go:28] interesting pod/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 08 15:37:37.776104 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.776025    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 08 15:37:37.972863 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.972833    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"]
Sep 08 15:37:37.975356 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:37:37.975328    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b28902e_c61b_437e_92bb_94e8bb7f9d94.slice/crio-e265a71db83908995fdef8cc2e06a0a9a166aba834e5bc2c5c57516ac0496c54 WatchSource:0}: Error finding container e265a71db83908995fdef8cc2e06a0a9a166aba834e5bc2c5c57516ac0496c54: Status 404 returned error can't find the container with id e265a71db83908995fdef8cc2e06a0a9a166aba834e5bc2c5c57516ac0496c54
Sep 08 15:37:37.977768 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.977736    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:37.977860 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.977817    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:37.977860 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:37.977847    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:37:38.045969 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:38.045936    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 08 15:37:38.046291 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:38.046182    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 08 15:37:38.642924 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:38.642886    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" event={"ID":"4b28902e-c61b-437e-92bb-94e8bb7f9d94","Type":"ContainerStarted","Data":"d8b73d76fe29534e700d1e672dc476740658eb682e7f53681313d7ffff9f6ca4"}
Sep 08 15:37:38.643558 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:38.643531    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" event={"ID":"4b28902e-c61b-437e-92bb-94e8bb7f9d94","Type":"ContainerStarted","Data":"e265a71db83908995fdef8cc2e06a0a9a166aba834e5bc2c5c57516ac0496c54"}
Sep 08 15:37:38.676065 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:38.676007    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" podStartSLOduration=1.675980418 podStartE2EDuration="1.675980418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:37:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:37:38.672699518 +0000 UTC m=+1706.758649551" watchObservedRunningTime="2026-09-08 15:37:38.675980418 +0000 UTC m=+1706.761930457"
Sep 08 15:37:38.775747 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:38.775680    2575 patch_prober.go:28] interesting pod/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 08 15:37:38.775956 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:38.775776    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 08 15:37:39.047644 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:39.047550    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 08 15:37:39.047644 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:39.047606    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 08 15:37:39.628640 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:39.628538    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:39.641523 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:39.641457    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:39.650857 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:39.650821    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:39.655563 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:39.655532    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:37:39.776082 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:39.776043    2575 patch_prober.go:28] interesting pod/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 08 15:37:39.776253 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:39.776115    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 08 15:37:39.876311 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:39.876275    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:40.046819 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:40.046718    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 08 15:37:40.046819 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:40.046782    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 08 15:37:40.775399 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:40.775355    2575 patch_prober.go:28] interesting pod/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 08 15:37:40.775897 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:40.775421    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 08 15:37:40.794186 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:40.794148    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:41.000771 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:41.000718    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:41.046736 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:41.046623    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 08 15:37:41.046736 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:41.046695    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 08 15:37:41.444506 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:41.444447    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:41.775117 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:41.775016    2575 patch_prober.go:28] interesting pod/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 08 15:37:41.775117 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:41.775077    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 08 15:37:42.046402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.046306    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 08 15:37:42.046402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.046380    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 08 15:37:42.481713 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.481679    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"]
Sep 08 15:37:42.484484 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.484461    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:42.499848 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.499823    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-ee0da5-serving-cert\""
Sep 08 15:37:42.506535 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.506510    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"]
Sep 08 15:37:42.545957 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.545919    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg6x2\" (UniqueName: \"kubernetes.io/projected/d76fa07b-cdef-4677-ad72-7dac7b918297-kube-api-access-pg6x2\") pod \"maas-api-e2e-api-b-ee0da5-547b954744-px7r4\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:42.546134 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.545972    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d76fa07b-cdef-4677-ad72-7dac7b918297-openshift-service-ca\") pod \"maas-api-e2e-api-b-ee0da5-547b954744-px7r4\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:42.546134 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.546082    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d76fa07b-cdef-4677-ad72-7dac7b918297-metrics-certs\") pod \"maas-api-e2e-api-b-ee0da5-547b954744-px7r4\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:42.546209 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.546151    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d76fa07b-cdef-4677-ad72-7dac7b918297-maas-api-tls\") pod \"maas-api-e2e-api-b-ee0da5-547b954744-px7r4\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:42.549713 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.549674    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:42.647141 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.647109    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d76fa07b-cdef-4677-ad72-7dac7b918297-metrics-certs\") pod \"maas-api-e2e-api-b-ee0da5-547b954744-px7r4\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:42.647336 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.647217    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d76fa07b-cdef-4677-ad72-7dac7b918297-maas-api-tls\") pod \"maas-api-e2e-api-b-ee0da5-547b954744-px7r4\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:42.647336 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.647255    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg6x2\" (UniqueName: \"kubernetes.io/projected/d76fa07b-cdef-4677-ad72-7dac7b918297-kube-api-access-pg6x2\") pod \"maas-api-e2e-api-b-ee0da5-547b954744-px7r4\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:42.647336 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.647290    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d76fa07b-cdef-4677-ad72-7dac7b918297-openshift-service-ca\") pod \"maas-api-e2e-api-b-ee0da5-547b954744-px7r4\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:42.647927 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.647905    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d76fa07b-cdef-4677-ad72-7dac7b918297-openshift-service-ca\") pod \"maas-api-e2e-api-b-ee0da5-547b954744-px7r4\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:42.649608 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.649552    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d76fa07b-cdef-4677-ad72-7dac7b918297-metrics-certs\") pod \"maas-api-e2e-api-b-ee0da5-547b954744-px7r4\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:42.649723 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.649702    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d76fa07b-cdef-4677-ad72-7dac7b918297-maas-api-tls\") pod \"maas-api-e2e-api-b-ee0da5-547b954744-px7r4\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:42.653543 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.653519    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:42.657669 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.657645    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg6x2\" (UniqueName: \"kubernetes.io/projected/d76fa07b-cdef-4677-ad72-7dac7b918297-kube-api-access-pg6x2\") pod \"maas-api-e2e-api-b-ee0da5-547b954744-px7r4\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:42.780235 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.780141    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:42.781293 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.781271    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:37:42.811982 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:42.811947    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:43.069997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:43.069911    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:43.070399 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:43.070376    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:43.072344 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:43.072316    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:37:43.108585 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:43.108554    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"]
Sep 08 15:37:43.112611 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:37:43.112584    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76fa07b_cdef_4677_ad72_7dac7b918297.slice/crio-19716ca61f2a33c44d363d79a2453a242f7810b6a30d43d56c60a8974347ffdb WatchSource:0}: Error finding container 19716ca61f2a33c44d363d79a2453a242f7810b6a30d43d56c60a8974347ffdb: Status 404 returned error can't find the container with id 19716ca61f2a33c44d363d79a2453a242f7810b6a30d43d56c60a8974347ffdb
Sep 08 15:37:43.171622 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:43.171582    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:43.267324 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:43.267281    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:43.549384 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:43.549349    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:43.674868 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:43.674832    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4" event={"ID":"d76fa07b-cdef-4677-ad72-7dac7b918297","Type":"ContainerStarted","Data":"19716ca61f2a33c44d363d79a2453a242f7810b6a30d43d56c60a8974347ffdb"}
Sep 08 15:37:43.935107 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:43.935069    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:44.682025 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:44.681988    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4" event={"ID":"d76fa07b-cdef-4677-ad72-7dac7b918297","Type":"ContainerStarted","Data":"0dd776a357d0274947a77ab8768876c2043817d8770777cdb2c83f0e6f82cc27"}
Sep 08 15:37:44.682461 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:44.682237    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:44.711964 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:44.711909    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4" podStartSLOduration=2.060128768 podStartE2EDuration="2.711893663s" totalImagesPullingTime="651.764895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:37:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:37:44.708288681 +0000 UTC m=+1712.794238723" watchObservedRunningTime="2026-09-08 15:37:44.711893663 +0000 UTC m=+1712.797843706"
Sep 08 15:37:46.845085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:46.845044    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:46.981008 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:46.980953    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:47.930262 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:47.930225    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"]
Sep 08 15:37:47.933381 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:47.933346    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:47.936669 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:47.936643    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-9dea65-serving-cert\""
Sep 08 15:37:47.959061 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:47.959022    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"]
Sep 08 15:37:48.007525 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.007454    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-metrics-certs\") pod \"maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:48.007680 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.007535    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7b6d\" (UniqueName: \"kubernetes.io/projected/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-kube-api-access-p7b6d\") pod \"maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:48.007680 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.007654    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-maas-api-tls\") pod \"maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:48.007806 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.007789    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:48.039547 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.039510    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:48.109537 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.109451    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:48.109759 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.109550    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-metrics-certs\") pod \"maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:48.109759 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.109598    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7b6d\" (UniqueName: \"kubernetes.io/projected/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-kube-api-access-p7b6d\") pod \"maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:48.109759 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.109663    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-maas-api-tls\") pod \"maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:48.110293 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.110260    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:48.112424 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.112396    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-maas-api-tls\") pod \"maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:48.112583 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.112426    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-metrics-certs\") pod \"maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:48.124427 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.124397    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7b6d\" (UniqueName: \"kubernetes.io/projected/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-kube-api-access-p7b6d\") pod \"maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:48.202229 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.202129    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:48.244913 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.244881    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:48.534337 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.534314    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"]
Sep 08 15:37:48.536471 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:37:48.536447    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7df27c42_13ea_46a1_a1e2_ad6a8343ddee.slice/crio-f7f01952c713d539d4be279d0020743f66f7b8460f88e3ad792464156ffc093f WatchSource:0}: Error finding container f7f01952c713d539d4be279d0020743f66f7b8460f88e3ad792464156ffc093f: Status 404 returned error can't find the container with id f7f01952c713d539d4be279d0020743f66f7b8460f88e3ad792464156ffc093f
Sep 08 15:37:48.708259 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:48.708228    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs" event={"ID":"7df27c42-13ea-46a1-a1e2-ad6a8343ddee","Type":"ContainerStarted","Data":"f7f01952c713d539d4be279d0020743f66f7b8460f88e3ad792464156ffc093f"}
Sep 08 15:37:49.028092 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:49.028056    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:49.210272 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:49.210170    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:37:49.715193 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:49.715143    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs" event={"ID":"7df27c42-13ea-46a1-a1e2-ad6a8343ddee","Type":"ContainerStarted","Data":"762f38c3d2776acb1d6348b32ae528fba51eb79e7aad4cf07a78c4a281f53bb1"}
Sep 08 15:37:49.715365 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:49.715250    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:37:49.740155 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:49.740077    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs" podStartSLOduration=2.416704356 podStartE2EDuration="2.740058582s" totalImagesPullingTime="323.354226ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:37:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:37:49.739333086 +0000 UTC m=+1717.825283128" watchObservedRunningTime="2026-09-08 15:37:49.740058582 +0000 UTC m=+1717.826008624"
Sep 08 15:37:50.693629 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:50.693599    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:37:54.754084 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:54.754044    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:37:55.728176 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:37:55.728146    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:38:01.339860 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:01.339823    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:03.650586 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:03.650544    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:05.796365 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:05.796323    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:06.813981 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:06.813944    2575 generic.go:356] "Generic (PLEG): container finished" podID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerID="d8b73d76fe29534e700d1e672dc476740658eb682e7f53681313d7ffff9f6ca4" exitCode=0
Sep 08 15:38:06.814396 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:06.814017    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" event={"ID":"4b28902e-c61b-437e-92bb-94e8bb7f9d94","Type":"ContainerDied","Data":"d8b73d76fe29534e700d1e672dc476740658eb682e7f53681313d7ffff9f6ca4"}
Sep 08 15:38:06.814579 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:06.814562    2575 scope.go:117] "RemoveContainer" containerID="d8b73d76fe29534e700d1e672dc476740658eb682e7f53681313d7ffff9f6ca4"
Sep 08 15:38:06.817409 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:06.817376    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:38:06.817689 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:06.817665    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:38:06.817869 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:06.817844    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:38:07.821320 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:07.821283    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" event={"ID":"4b28902e-c61b-437e-92bb-94e8bb7f9d94","Type":"ContainerStarted","Data":"23bb34acd5d995ad1bb3142f6a63881c5e751801afc692c21316664460aeb36d"}
Sep 08 15:38:08.628718 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:08.628686    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:38:08.630065 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:08.630037    2575 patch_prober.go:28] interesting pod/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 08 15:38:08.630188 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:08.630085    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 08 15:38:09.057966 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:09.057886    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:09.629026 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:09.628991    2575 patch_prober.go:28] interesting pod/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 08 15:38:09.629203 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:09.629043    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 08 15:38:09.651014 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:09.650984    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:38:10.628792 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:10.628753    2575 patch_prober.go:28] interesting pod/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 08 15:38:10.629182 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:10.628809    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 08 15:38:11.133021 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:11.132987    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:11.628603 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:11.628567    2575 patch_prober.go:28] interesting pod/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 08 15:38:11.628791 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:11.628673    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 08 15:38:12.628843 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:12.628806    2575 patch_prober.go:28] interesting pod/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 08 15:38:12.629219 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:12.628870    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 08 15:38:13.022618 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:13.022530    2575 scope.go:117] "RemoveContainer" containerID="1d291cc784731b104a12c64dbf65d847df88fb8cf2792a342a91c38dfb11a700"
Sep 08 15:38:13.404141 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:13.404095    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:13.632696 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:13.632668    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:38:13.633701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:13.633681    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:38:16.841166 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:16.841132    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:19.019389 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:19.019351    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:19.119742 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:19.119684    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:24.752633 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:24.752589    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"]
Sep 08 15:38:24.753032 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:24.752865    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8" podUID="7010f537-f8dd-4aef-8c86-551ecd79c7e0" containerName="maas-api" containerID="cri-o://eb5628e89cc6a480f48220ac555a68c0bb8a77a06e276eb17fc596d040f3980c" gracePeriod=30
Sep 08 15:38:24.807919 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:24.807883    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:24.917515 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:24.917463    2575 generic.go:356] "Generic (PLEG): container finished" podID="7010f537-f8dd-4aef-8c86-551ecd79c7e0" containerID="eb5628e89cc6a480f48220ac555a68c0bb8a77a06e276eb17fc596d040f3980c" exitCode=0
Sep 08 15:38:24.917681 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:24.917541    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8" event={"ID":"7010f537-f8dd-4aef-8c86-551ecd79c7e0","Type":"ContainerDied","Data":"eb5628e89cc6a480f48220ac555a68c0bb8a77a06e276eb17fc596d040f3980c"}
Sep 08 15:38:24.950956 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:24.950919    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:25.016530 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.016454    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:38:25.113307 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.113275    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7p7b\" (UniqueName: \"kubernetes.io/projected/7010f537-f8dd-4aef-8c86-551ecd79c7e0-kube-api-access-f7p7b\") pod \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") "
Sep 08 15:38:25.113532 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.113362    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-metrics-certs\") pod \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") "
Sep 08 15:38:25.113532 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.113416    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7010f537-f8dd-4aef-8c86-551ecd79c7e0-openshift-service-ca\") pod \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") "
Sep 08 15:38:25.113532 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.113486    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-maas-api-tls\") pod \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\" (UID: \"7010f537-f8dd-4aef-8c86-551ecd79c7e0\") "
Sep 08 15:38:25.113883 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.113852    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7010f537-f8dd-4aef-8c86-551ecd79c7e0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7010f537-f8dd-4aef-8c86-551ecd79c7e0" (UID: "7010f537-f8dd-4aef-8c86-551ecd79c7e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:38:25.116338 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.116286    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7010f537-f8dd-4aef-8c86-551ecd79c7e0" (UID: "7010f537-f8dd-4aef-8c86-551ecd79c7e0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:38:25.116338 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.116362    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7010f537-f8dd-4aef-8c86-551ecd79c7e0-kube-api-access-f7p7b" (OuterVolumeSpecName: "kube-api-access-f7p7b") pod "7010f537-f8dd-4aef-8c86-551ecd79c7e0" (UID: "7010f537-f8dd-4aef-8c86-551ecd79c7e0"). InnerVolumeSpecName "kube-api-access-f7p7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:38:25.116637 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.116520    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:25.117423 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.117386    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7010f537-f8dd-4aef-8c86-551ecd79c7e0" (UID: "7010f537-f8dd-4aef-8c86-551ecd79c7e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:38:25.214856 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.214821    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7p7b\" (UniqueName: \"kubernetes.io/projected/7010f537-f8dd-4aef-8c86-551ecd79c7e0-kube-api-access-f7p7b\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:25.214856 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.214857    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-metrics-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:25.215065 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.214871    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7010f537-f8dd-4aef-8c86-551ecd79c7e0-openshift-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:25.215065 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.214883    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7010f537-f8dd-4aef-8c86-551ecd79c7e0-maas-api-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:25.220807 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.220779    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:25.570043 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.570003    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:25.924240 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.924206    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"
Sep 08 15:38:25.924240 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.924227    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8" event={"ID":"7010f537-f8dd-4aef-8c86-551ecd79c7e0","Type":"ContainerDied","Data":"e748ad70dc5c149c82fb9192e950972408a7883e8fdeda7a89820d5d0d2d4576"}
Sep 08 15:38:25.924787 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.924291    2575 scope.go:117] "RemoveContainer" containerID="eb5628e89cc6a480f48220ac555a68c0bb8a77a06e276eb17fc596d040f3980c"
Sep 08 15:38:25.947214 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.947181    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"]
Sep 08 15:38:25.951265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:25.951243    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-61a1d6-7697757df4-lbbg8"]
Sep 08 15:38:26.432668 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:26.432635    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7010f537-f8dd-4aef-8c86-551ecd79c7e0" path="/var/lib/kubelet/pods/7010f537-f8dd-4aef-8c86-551ecd79c7e0/volumes"
Sep 08 15:38:32.262669 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:32.262634    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:32.366973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:32.366942    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:32.532377 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:32.532299    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"]
Sep 08 15:38:32.532551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:32.532505    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerName="istio-proxy" containerID="cri-o://570d58dc5c8e82eb72f868f0894da77853b6ead596bdf6af11458075892e7004" gracePeriod=30
Sep 08 15:38:32.568956 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:32.568925    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:33.256375 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:33.256340    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:33.568786 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:33.568705    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:38:37.161983 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.161949    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"]
Sep 08 15:38:37.162396 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.162382    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7010f537-f8dd-4aef-8c86-551ecd79c7e0" containerName="maas-api"
Sep 08 15:38:37.162396 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.162396    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7010f537-f8dd-4aef-8c86-551ecd79c7e0" containerName="maas-api"
Sep 08 15:38:37.162483 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.162464    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7010f537-f8dd-4aef-8c86-551ecd79c7e0" containerName="maas-api"
Sep 08 15:38:37.166053 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.166038    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.171467 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.171445    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-4a2c80\"/\"kube-root-ca.crt\""
Sep 08 15:38:37.171597 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.171475    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-4a2c80\"/\"openshift-service-ca.crt\""
Sep 08 15:38:37.172452 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.172434    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-4a2c80\"/\"default-dockercfg-blhv8\""
Sep 08 15:38:37.172564 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.172473    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-4a2c80\"/\"ipp-route-4a2c80-kserve-self-signed-certs\""
Sep 08 15:38:37.184021 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.183998    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"]
Sep 08 15:38:37.227299 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.227080    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:37.243092 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.243062    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gqdz\" (UniqueName: \"kubernetes.io/projected/a348d026-4378-47a3-a836-97d173458c5b-kube-api-access-9gqdz\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.243092 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.243100    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-tmp-dir\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.243276 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.243173    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-model-cache\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.243276 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.243236    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a348d026-4378-47a3-a836-97d173458c5b-tls-certs\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.243276 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.243266    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-home\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.243401 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.243308    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-dshm\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.344048 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.344007    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gqdz\" (UniqueName: \"kubernetes.io/projected/a348d026-4378-47a3-a836-97d173458c5b-kube-api-access-9gqdz\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.344048 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.344051    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-tmp-dir\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.344402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.344101    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-model-cache\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.344402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.344155    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a348d026-4378-47a3-a836-97d173458c5b-tls-certs\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.344402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.344222    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-home\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.344402 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.344283    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-dshm\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.344581 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.344560    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-tmp-dir\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.344667 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.344601    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-home\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.344667 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.344631    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-model-cache\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.346478 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.346453    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-dshm\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.346668 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.346650    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a348d026-4378-47a3-a836-97d173458c5b-tls-certs\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.354146 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.354123    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gqdz\" (UniqueName: \"kubernetes.io/projected/a348d026-4378-47a3-a836-97d173458c5b-kube-api-access-9gqdz\") pod \"ipp-route-4a2c80-kserve-89b558f87-vd9pk\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") " pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.475848 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.475752    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:37.758285 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.758256    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:38:37.761861 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.761839    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"]
Sep 08 15:38:37.763716 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:38:37.763692    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda348d026_4378_47a3_a836_97d173458c5b.slice/crio-b5af01f70d4918bd3222856b835753da481125e751266daa12f730a50fe68533 WatchSource:0}: Error finding container b5af01f70d4918bd3222856b835753da481125e751266daa12f730a50fe68533: Status 404 returned error can't find the container with id b5af01f70d4918bd3222856b835753da481125e751266daa12f730a50fe68533
Sep 08 15:38:37.853676 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.853643    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tb9g\" (UniqueName: \"kubernetes.io/projected/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-kube-api-access-5tb9g\") pod \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") "
Sep 08 15:38:37.853676 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.853678    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-credential-socket\") pod \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") "
Sep 08 15:38:37.853927 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.853723    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-workload-socket\") pod \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") "
Sep 08 15:38:37.853927 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.853761    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-data\") pod \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") "
Sep 08 15:38:37.853927 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.853797    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-workload-certs\") pod \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") "
Sep 08 15:38:37.853927 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.853835    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istiod-ca-cert\") pod \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") "
Sep 08 15:38:37.853927 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.853856    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-podinfo\") pod \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") "
Sep 08 15:38:37.853927 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.853923    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-token\") pod \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") "
Sep 08 15:38:37.854244 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.853966    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-envoy\") pod \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\" (UID: \"2ff6cd8a-5ffc-4392-a9be-544a15678ed1\") "
Sep 08 15:38:37.854244 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.853961    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2ff6cd8a-5ffc-4392-a9be-544a15678ed1" (UID: "2ff6cd8a-5ffc-4392-a9be-544a15678ed1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:38:37.854353 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.854236    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2ff6cd8a-5ffc-4392-a9be-544a15678ed1" (UID: "2ff6cd8a-5ffc-4392-a9be-544a15678ed1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:38:37.854353 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.854246    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2ff6cd8a-5ffc-4392-a9be-544a15678ed1" (UID: "2ff6cd8a-5ffc-4392-a9be-544a15678ed1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:38:37.854353 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.854298    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-data" (OuterVolumeSpecName: "istio-data") pod "2ff6cd8a-5ffc-4392-a9be-544a15678ed1" (UID: "2ff6cd8a-5ffc-4392-a9be-544a15678ed1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:38:37.854478 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.854392    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2ff6cd8a-5ffc-4392-a9be-544a15678ed1" (UID: "2ff6cd8a-5ffc-4392-a9be-544a15678ed1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:38:37.854740 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.854717    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:37.854839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.854744    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:37.854839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.854759    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:37.854839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.854773    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:37.854839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.854787    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:37.856078 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.856033    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2ff6cd8a-5ffc-4392-a9be-544a15678ed1" (UID: "2ff6cd8a-5ffc-4392-a9be-544a15678ed1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:38:37.856182 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.856111    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-token" (OuterVolumeSpecName: "istio-token") pod "2ff6cd8a-5ffc-4392-a9be-544a15678ed1" (UID: "2ff6cd8a-5ffc-4392-a9be-544a15678ed1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:38:37.856182 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.856127    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2ff6cd8a-5ffc-4392-a9be-544a15678ed1" (UID: "2ff6cd8a-5ffc-4392-a9be-544a15678ed1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:38:37.856182 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.856134    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-kube-api-access-5tb9g" (OuterVolumeSpecName: "kube-api-access-5tb9g") pod "2ff6cd8a-5ffc-4392-a9be-544a15678ed1" (UID: "2ff6cd8a-5ffc-4392-a9be-544a15678ed1"). InnerVolumeSpecName "kube-api-access-5tb9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:38:37.955877 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.955829    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:37.955877 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.955868    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tb9g\" (UniqueName: \"kubernetes.io/projected/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-kube-api-access-5tb9g\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:37.955877 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.955882    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:37.956121 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.955893    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ff6cd8a-5ffc-4392-a9be-544a15678ed1-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:37.994506 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.994452    2575 generic.go:356] "Generic (PLEG): container finished" podID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerID="570d58dc5c8e82eb72f868f0894da77853b6ead596bdf6af11458075892e7004" exitCode=0
Sep 08 15:38:37.994673 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.994562    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"
Sep 08 15:38:37.994673 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.994589    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" event={"ID":"2ff6cd8a-5ffc-4392-a9be-544a15678ed1","Type":"ContainerDied","Data":"570d58dc5c8e82eb72f868f0894da77853b6ead596bdf6af11458075892e7004"}
Sep 08 15:38:37.994673 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.994645    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm" event={"ID":"2ff6cd8a-5ffc-4392-a9be-544a15678ed1","Type":"ContainerDied","Data":"a25a23cf091c937c0e177199a9d0c8f48e31f6bb5cf37499771e90f636c4bf26"}
Sep 08 15:38:37.994673 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.994668    2575 scope.go:117] "RemoveContainer" containerID="570d58dc5c8e82eb72f868f0894da77853b6ead596bdf6af11458075892e7004"
Sep 08 15:38:37.996610 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.996523    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk" event={"ID":"a348d026-4378-47a3-a836-97d173458c5b","Type":"ContainerStarted","Data":"b574a020a69abe8cb49866e9252f6e1400d683ae157b453feeffce92db3be7aa"}
Sep 08 15:38:37.996610 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:37.996569    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk" event={"ID":"a348d026-4378-47a3-a836-97d173458c5b","Type":"ContainerStarted","Data":"b5af01f70d4918bd3222856b835753da481125e751266daa12f730a50fe68533"}
Sep 08 15:38:38.004500 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:38.004466    2575 scope.go:117] "RemoveContainer" containerID="e037ea345efdbfbc248d2d2ea556c0b8a3ca58acf4676285fef2b490c07116b8"
Sep 08 15:38:38.013595 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:38.013470    2575 scope.go:117] "RemoveContainer" containerID="570d58dc5c8e82eb72f868f0894da77853b6ead596bdf6af11458075892e7004"
Sep 08 15:38:38.013847 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:38:38.013828    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"570d58dc5c8e82eb72f868f0894da77853b6ead596bdf6af11458075892e7004\": container with ID starting with 570d58dc5c8e82eb72f868f0894da77853b6ead596bdf6af11458075892e7004 not found: ID does not exist" containerID="570d58dc5c8e82eb72f868f0894da77853b6ead596bdf6af11458075892e7004"
Sep 08 15:38:38.013892 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:38.013857    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"570d58dc5c8e82eb72f868f0894da77853b6ead596bdf6af11458075892e7004"} err="failed to get container status \"570d58dc5c8e82eb72f868f0894da77853b6ead596bdf6af11458075892e7004\": rpc error: code = NotFound desc = could not find container \"570d58dc5c8e82eb72f868f0894da77853b6ead596bdf6af11458075892e7004\": container with ID starting with 570d58dc5c8e82eb72f868f0894da77853b6ead596bdf6af11458075892e7004 not found: ID does not exist"
Sep 08 15:38:38.013892 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:38.013878    2575 scope.go:117] "RemoveContainer" containerID="e037ea345efdbfbc248d2d2ea556c0b8a3ca58acf4676285fef2b490c07116b8"
Sep 08 15:38:38.014152 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:38:38.014131    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e037ea345efdbfbc248d2d2ea556c0b8a3ca58acf4676285fef2b490c07116b8\": container with ID starting with e037ea345efdbfbc248d2d2ea556c0b8a3ca58acf4676285fef2b490c07116b8 not found: ID does not exist" containerID="e037ea345efdbfbc248d2d2ea556c0b8a3ca58acf4676285fef2b490c07116b8"
Sep 08 15:38:38.014206 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:38.014158    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e037ea345efdbfbc248d2d2ea556c0b8a3ca58acf4676285fef2b490c07116b8"} err="failed to get container status \"e037ea345efdbfbc248d2d2ea556c0b8a3ca58acf4676285fef2b490c07116b8\": rpc error: code = NotFound desc = could not find container \"e037ea345efdbfbc248d2d2ea556c0b8a3ca58acf4676285fef2b490c07116b8\": container with ID starting with e037ea345efdbfbc248d2d2ea556c0b8a3ca58acf4676285fef2b490c07116b8 not found: ID does not exist"
Sep 08 15:38:38.016065 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:38.016022    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk" podStartSLOduration=1.016009655 podStartE2EDuration="1.016009655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:38:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:38:38.014922483 +0000 UTC m=+1766.100872539" watchObservedRunningTime="2026-09-08 15:38:38.016009655 +0000 UTC m=+1766.101959714"
Sep 08 15:38:38.031840 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:38.031789    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"]
Sep 08 15:38:38.033930 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:38.033909    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-61a1d6-openshift-default-79599fc894-rnlzm"]
Sep 08 15:38:38.433061 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:38.433015    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" path="/var/lib/kubelet/pods/2ff6cd8a-5ffc-4392-a9be-544a15678ed1/volumes"
Sep 08 15:38:47.476743 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:47.476699    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:47.477156 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:47.476764    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:47.498826 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:47.498795    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:48.062702 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.062670    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:38:48.642692 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.642650    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"]
Sep 08 15:38:48.643108 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.642961    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4" podUID="d76fa07b-cdef-4677-ad72-7dac7b918297" containerName="maas-api" containerID="cri-o://0dd776a357d0274947a77ab8768876c2043817d8770777cdb2c83f0e6f82cc27" gracePeriod=30
Sep 08 15:38:48.675293 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.675259    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:48.753626 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.753569    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:48.852450 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.852409    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:48.905312 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.905256    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:38:48.921753 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.921724    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:48.978114 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.978082    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pg6x2\" (UniqueName: \"kubernetes.io/projected/d76fa07b-cdef-4677-ad72-7dac7b918297-kube-api-access-pg6x2\") pod \"d76fa07b-cdef-4677-ad72-7dac7b918297\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") "
Sep 08 15:38:48.978285 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.978152    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d76fa07b-cdef-4677-ad72-7dac7b918297-openshift-service-ca\") pod \"d76fa07b-cdef-4677-ad72-7dac7b918297\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") "
Sep 08 15:38:48.978285 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.978217    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d76fa07b-cdef-4677-ad72-7dac7b918297-metrics-certs\") pod \"d76fa07b-cdef-4677-ad72-7dac7b918297\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") "
Sep 08 15:38:48.978285 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.978278    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d76fa07b-cdef-4677-ad72-7dac7b918297-maas-api-tls\") pod \"d76fa07b-cdef-4677-ad72-7dac7b918297\" (UID: \"d76fa07b-cdef-4677-ad72-7dac7b918297\") "
Sep 08 15:38:48.978564 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.978538    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d76fa07b-cdef-4677-ad72-7dac7b918297-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d76fa07b-cdef-4677-ad72-7dac7b918297" (UID: "d76fa07b-cdef-4677-ad72-7dac7b918297"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:38:48.978730 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.978714    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d76fa07b-cdef-4677-ad72-7dac7b918297-openshift-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:48.980300 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.980282    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d76fa07b-cdef-4677-ad72-7dac7b918297-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d76fa07b-cdef-4677-ad72-7dac7b918297" (UID: "d76fa07b-cdef-4677-ad72-7dac7b918297"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:38:48.980377 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.980341    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d76fa07b-cdef-4677-ad72-7dac7b918297-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d76fa07b-cdef-4677-ad72-7dac7b918297" (UID: "d76fa07b-cdef-4677-ad72-7dac7b918297"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:38:48.980416 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:48.980373    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d76fa07b-cdef-4677-ad72-7dac7b918297-kube-api-access-pg6x2" (OuterVolumeSpecName: "kube-api-access-pg6x2") pod "d76fa07b-cdef-4677-ad72-7dac7b918297" (UID: "d76fa07b-cdef-4677-ad72-7dac7b918297"). InnerVolumeSpecName "kube-api-access-pg6x2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:38:49.061601 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:49.061565    2575 generic.go:356] "Generic (PLEG): container finished" podID="d76fa07b-cdef-4677-ad72-7dac7b918297" containerID="0dd776a357d0274947a77ab8768876c2043817d8770777cdb2c83f0e6f82cc27" exitCode=0
Sep 08 15:38:49.061807 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:49.061627    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"
Sep 08 15:38:49.061807 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:49.061709    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4" event={"ID":"d76fa07b-cdef-4677-ad72-7dac7b918297","Type":"ContainerDied","Data":"0dd776a357d0274947a77ab8768876c2043817d8770777cdb2c83f0e6f82cc27"}
Sep 08 15:38:49.061807 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:49.061756    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4" event={"ID":"d76fa07b-cdef-4677-ad72-7dac7b918297","Type":"ContainerDied","Data":"19716ca61f2a33c44d363d79a2453a242f7810b6a30d43d56c60a8974347ffdb"}
Sep 08 15:38:49.061807 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:49.061785    2575 scope.go:117] "RemoveContainer" containerID="0dd776a357d0274947a77ab8768876c2043817d8770777cdb2c83f0e6f82cc27"
Sep 08 15:38:49.079284 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:49.079258    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg6x2\" (UniqueName: \"kubernetes.io/projected/d76fa07b-cdef-4677-ad72-7dac7b918297-kube-api-access-pg6x2\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:49.079284 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:49.079285    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d76fa07b-cdef-4677-ad72-7dac7b918297-metrics-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:49.079451 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:49.079298    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d76fa07b-cdef-4677-ad72-7dac7b918297-maas-api-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:38:49.079791 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:49.079769    2575 scope.go:117] "RemoveContainer" containerID="0dd776a357d0274947a77ab8768876c2043817d8770777cdb2c83f0e6f82cc27"
Sep 08 15:38:49.080084 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:38:49.080063    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dd776a357d0274947a77ab8768876c2043817d8770777cdb2c83f0e6f82cc27\": container with ID starting with 0dd776a357d0274947a77ab8768876c2043817d8770777cdb2c83f0e6f82cc27 not found: ID does not exist" containerID="0dd776a357d0274947a77ab8768876c2043817d8770777cdb2c83f0e6f82cc27"
Sep 08 15:38:49.080199 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:49.080093    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dd776a357d0274947a77ab8768876c2043817d8770777cdb2c83f0e6f82cc27"} err="failed to get container status \"0dd776a357d0274947a77ab8768876c2043817d8770777cdb2c83f0e6f82cc27\": rpc error: code = NotFound desc = could not find container \"0dd776a357d0274947a77ab8768876c2043817d8770777cdb2c83f0e6f82cc27\": container with ID starting with 0dd776a357d0274947a77ab8768876c2043817d8770777cdb2c83f0e6f82cc27 not found: ID does not exist"
Sep 08 15:38:49.093289 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:49.093260    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"]
Sep 08 15:38:49.098978 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:49.098954    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ee0da5-547b954744-px7r4"]
Sep 08 15:38:49.283714 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:49.283628    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:50.431737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:50.431702    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d76fa07b-cdef-4677-ad72-7dac7b918297" path="/var/lib/kubelet/pods/d76fa07b-cdef-4677-ad72-7dac7b918297/volumes"
Sep 08 15:38:53.345170 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:53.345136    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:38:56.179131 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:56.179096    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:56.260460 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:56.260424    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:56.455446 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:56.455364    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"]
Sep 08 15:38:56.455664 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:56.455639    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerName="istio-proxy" containerID="cri-o://23bb34acd5d995ad1bb3142f6a63881c5e751801afc692c21316664460aeb36d" gracePeriod=30
Sep 08 15:38:56.481415 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:56.481373    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:57.024892 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:57.024854    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:38:58.635715 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:58.635676    2575 patch_prober.go:28] interesting pod/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 15:38:58.636066 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:38:58.635735    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 15:39:01.711519 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.711482    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:39:01.811550 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.811447    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86r4g\" (UniqueName: \"kubernetes.io/projected/4b28902e-c61b-437e-92bb-94e8bb7f9d94-kube-api-access-86r4g\") pod \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") "
Sep 08 15:39:01.811550 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.811483    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-workload-certs\") pod \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") "
Sep 08 15:39:01.811789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.811557    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istiod-ca-cert\") pod \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") "
Sep 08 15:39:01.811789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.811603    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-token\") pod \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") "
Sep 08 15:39:01.811789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.811636    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-envoy\") pod \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") "
Sep 08 15:39:01.811789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.811664    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-data\") pod \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") "
Sep 08 15:39:01.811789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.811717    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-workload-socket\") pod \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") "
Sep 08 15:39:01.811789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.811759    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-credential-socket\") pod \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") "
Sep 08 15:39:01.812090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.811800    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-podinfo\") pod \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\" (UID: \"4b28902e-c61b-437e-92bb-94e8bb7f9d94\") "
Sep 08 15:39:01.812090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.811839    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4b28902e-c61b-437e-92bb-94e8bb7f9d94" (UID: "4b28902e-c61b-437e-92bb-94e8bb7f9d94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:39:01.812090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.811992    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-data" (OuterVolumeSpecName: "istio-data") pod "4b28902e-c61b-437e-92bb-94e8bb7f9d94" (UID: "4b28902e-c61b-437e-92bb-94e8bb7f9d94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:39:01.812090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.812007    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4b28902e-c61b-437e-92bb-94e8bb7f9d94" (UID: "4b28902e-c61b-437e-92bb-94e8bb7f9d94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:39:01.812090 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.812085    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4b28902e-c61b-437e-92bb-94e8bb7f9d94" (UID: "4b28902e-c61b-437e-92bb-94e8bb7f9d94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:39:01.812368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.812103    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4b28902e-c61b-437e-92bb-94e8bb7f9d94" (UID: "4b28902e-c61b-437e-92bb-94e8bb7f9d94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:39:01.812368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.812347    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:01.812464 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.812372    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:01.812464 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.812386    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:01.812464 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.812401    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:01.812464 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.812414    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:01.813788 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.813768    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b28902e-c61b-437e-92bb-94e8bb7f9d94-kube-api-access-86r4g" (OuterVolumeSpecName: "kube-api-access-86r4g") pod "4b28902e-c61b-437e-92bb-94e8bb7f9d94" (UID: "4b28902e-c61b-437e-92bb-94e8bb7f9d94"). InnerVolumeSpecName "kube-api-access-86r4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:39:01.814041 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.814019    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4b28902e-c61b-437e-92bb-94e8bb7f9d94" (UID: "4b28902e-c61b-437e-92bb-94e8bb7f9d94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:39:01.814150 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.814132    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4b28902e-c61b-437e-92bb-94e8bb7f9d94" (UID: "4b28902e-c61b-437e-92bb-94e8bb7f9d94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:39:01.814234 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.814219    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-token" (OuterVolumeSpecName: "istio-token") pod "4b28902e-c61b-437e-92bb-94e8bb7f9d94" (UID: "4b28902e-c61b-437e-92bb-94e8bb7f9d94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:39:01.913433 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.913399    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86r4g\" (UniqueName: \"kubernetes.io/projected/4b28902e-c61b-437e-92bb-94e8bb7f9d94-kube-api-access-86r4g\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:01.913433 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.913428    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:01.913433 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.913438    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:01.913663 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:01.913447    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b28902e-c61b-437e-92bb-94e8bb7f9d94-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:02.133819 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:02.133783    2575 generic.go:356] "Generic (PLEG): container finished" podID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerID="23bb34acd5d995ad1bb3142f6a63881c5e751801afc692c21316664460aeb36d" exitCode=0
Sep 08 15:39:02.133963 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:02.133835    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"
Sep 08 15:39:02.133963 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:02.133850    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" event={"ID":"4b28902e-c61b-437e-92bb-94e8bb7f9d94","Type":"ContainerDied","Data":"23bb34acd5d995ad1bb3142f6a63881c5e751801afc692c21316664460aeb36d"}
Sep 08 15:39:02.133963 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:02.133905    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb" event={"ID":"4b28902e-c61b-437e-92bb-94e8bb7f9d94","Type":"ContainerDied","Data":"e265a71db83908995fdef8cc2e06a0a9a166aba834e5bc2c5c57516ac0496c54"}
Sep 08 15:39:02.133963 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:02.133929    2575 scope.go:117] "RemoveContainer" containerID="23bb34acd5d995ad1bb3142f6a63881c5e751801afc692c21316664460aeb36d"
Sep 08 15:39:02.143782 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:02.143761    2575 scope.go:117] "RemoveContainer" containerID="d8b73d76fe29534e700d1e672dc476740658eb682e7f53681313d7ffff9f6ca4"
Sep 08 15:39:02.151832 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:02.151812    2575 scope.go:117] "RemoveContainer" containerID="23bb34acd5d995ad1bb3142f6a63881c5e751801afc692c21316664460aeb36d"
Sep 08 15:39:02.152116 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:39:02.152098    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23bb34acd5d995ad1bb3142f6a63881c5e751801afc692c21316664460aeb36d\": container with ID starting with 23bb34acd5d995ad1bb3142f6a63881c5e751801afc692c21316664460aeb36d not found: ID does not exist" containerID="23bb34acd5d995ad1bb3142f6a63881c5e751801afc692c21316664460aeb36d"
Sep 08 15:39:02.152162 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:02.152124    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23bb34acd5d995ad1bb3142f6a63881c5e751801afc692c21316664460aeb36d"} err="failed to get container status \"23bb34acd5d995ad1bb3142f6a63881c5e751801afc692c21316664460aeb36d\": rpc error: code = NotFound desc = could not find container \"23bb34acd5d995ad1bb3142f6a63881c5e751801afc692c21316664460aeb36d\": container with ID starting with 23bb34acd5d995ad1bb3142f6a63881c5e751801afc692c21316664460aeb36d not found: ID does not exist"
Sep 08 15:39:02.152162 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:02.152140    2575 scope.go:117] "RemoveContainer" containerID="d8b73d76fe29534e700d1e672dc476740658eb682e7f53681313d7ffff9f6ca4"
Sep 08 15:39:02.152379 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:39:02.152357    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8b73d76fe29534e700d1e672dc476740658eb682e7f53681313d7ffff9f6ca4\": container with ID starting with d8b73d76fe29534e700d1e672dc476740658eb682e7f53681313d7ffff9f6ca4 not found: ID does not exist" containerID="d8b73d76fe29534e700d1e672dc476740658eb682e7f53681313d7ffff9f6ca4"
Sep 08 15:39:02.152439 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:02.152389    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b73d76fe29534e700d1e672dc476740658eb682e7f53681313d7ffff9f6ca4"} err="failed to get container status \"d8b73d76fe29534e700d1e672dc476740658eb682e7f53681313d7ffff9f6ca4\": rpc error: code = NotFound desc = could not find container \"d8b73d76fe29534e700d1e672dc476740658eb682e7f53681313d7ffff9f6ca4\": container with ID starting with d8b73d76fe29534e700d1e672dc476740658eb682e7f53681313d7ffff9f6ca4 not found: ID does not exist"
Sep 08 15:39:02.158642 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:02.158619    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"]
Sep 08 15:39:02.164807 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:02.164786    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ee0da5-openshift-default-5d59b4d47d-stpkb"]
Sep 08 15:39:02.432098 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:02.432021    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" path="/var/lib/kubelet/pods/4b28902e-c61b-437e-92bb-94e8bb7f9d94/volumes"
Sep 08 15:39:07.538154 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:07.537890    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:07.632967 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:07.632925    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:07.739713 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:07.739651    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:07.935973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:07.935939    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:11.247022 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:11.246984    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:39:12.460266 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:12.460235    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:39:12.468567 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:12.468538    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:39:18.162042 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:18.162004    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:24.455821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:24.455787    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"]
Sep 08 15:39:24.456199 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:24.456055    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerName="istio-proxy" containerID="cri-o://d8f4f4531382642356626409f3b332411f61c0ef02a721bfa45775091ef9c723" gracePeriod=30
Sep 08 15:39:25.879071 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:25.879030    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:25.978828 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:25.978788    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:28.735502 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:28.735462    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:29.034242 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034161    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"]
Sep 08 15:39:29.034685 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034666    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerName="istio-proxy"
Sep 08 15:39:29.034789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034687    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerName="istio-proxy"
Sep 08 15:39:29.034789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034735    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerName="istio-proxy"
Sep 08 15:39:29.034789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034744    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerName="istio-proxy"
Sep 08 15:39:29.034789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034753    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerName="istio-proxy"
Sep 08 15:39:29.034789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034762    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerName="istio-proxy"
Sep 08 15:39:29.034789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034776    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d76fa07b-cdef-4677-ad72-7dac7b918297" containerName="maas-api"
Sep 08 15:39:29.034789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034784    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d76fa07b-cdef-4677-ad72-7dac7b918297" containerName="maas-api"
Sep 08 15:39:29.035129 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034879    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerName="istio-proxy"
Sep 08 15:39:29.035129 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034892    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerName="istio-proxy"
Sep 08 15:39:29.035129 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034905    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerName="istio-proxy"
Sep 08 15:39:29.035129 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034917    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ff6cd8a-5ffc-4392-a9be-544a15678ed1" containerName="istio-proxy"
Sep 08 15:39:29.035129 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.034928    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d76fa07b-cdef-4677-ad72-7dac7b918297" containerName="maas-api"
Sep 08 15:39:29.038163 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.038144    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.041774 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.041754    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-d97358-openshift-default-dockercfg-sssr2\""
Sep 08 15:39:29.059724 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.059702    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"]
Sep 08 15:39:29.070990 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.070968    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:29.187042 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.187003    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rzgl\" (UniqueName: \"kubernetes.io/projected/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-kube-api-access-2rzgl\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.187265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.187050    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.187265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.187160    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-envoy\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.187265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.187214    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-workload-socket\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.187430 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.187310    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-credential-socket\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.187430 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.187375    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-data\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.187430 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.187415    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-podinfo\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.187649 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.187448    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-workload-certs\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.187649 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.187570    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-token\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.234274 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.234247    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:29.288450 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.288379    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-credential-socket\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.288450 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.288418    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-data\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.288651 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.288551    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-podinfo\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.288651 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.288595    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-workload-certs\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.288764 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.288653    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-token\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.288764 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.288724    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rzgl\" (UniqueName: \"kubernetes.io/projected/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-kube-api-access-2rzgl\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.288869 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.288763    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.288869 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.288857    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-envoy\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.288991 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.288891    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-credential-socket\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.288991 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.288902    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-workload-socket\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.288991 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.288971    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-workload-certs\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.289224 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.289199    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-workload-socket\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.289396 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.289270    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-data\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.289590 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.289571    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.291071 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.291050    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-envoy\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.291152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.291085    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-podinfo\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.297780 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.297752    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-token\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.297887 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.297817    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rzgl\" (UniqueName: \"kubernetes.io/projected/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-kube-api-access-2rzgl\") pod \"e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") " pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.348524 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.348484    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:29.636925 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.636895    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"]
Sep 08 15:39:29.638681 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:39:29.638640    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd6d10e_779b_4f2f_94bd_5333cc0093b0.slice/crio-c824d19f6ea4542fd2859671422b75efe6f5968ae1ba08f9a5aad28d63a601ac WatchSource:0}: Error finding container c824d19f6ea4542fd2859671422b75efe6f5968ae1ba08f9a5aad28d63a601ac: Status 404 returned error can't find the container with id c824d19f6ea4542fd2859671422b75efe6f5968ae1ba08f9a5aad28d63a601ac
Sep 08 15:39:29.641264 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.641229    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:39:29.641344 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.641301    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:39:29.641344 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.641329    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:39:29.688032 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.688011    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:39:29.794128 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.794088    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-token\") pod \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") "
Sep 08 15:39:29.794621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.794169    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-credential-socket\") pod \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") "
Sep 08 15:39:29.794621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.794270    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-workload-socket\") pod \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") "
Sep 08 15:39:29.794621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.794303    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhr2m\" (UniqueName: \"kubernetes.io/projected/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-kube-api-access-fhr2m\") pod \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") "
Sep 08 15:39:29.794621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.794353    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-podinfo\") pod \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") "
Sep 08 15:39:29.794621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.794391    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-data\") pod \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") "
Sep 08 15:39:29.794621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.794461    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-envoy\") pod \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") "
Sep 08 15:39:29.794621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.794517    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7ff18c8-3caf-4048-b7d8-3c5699caadcc" (UID: "d7ff18c8-3caf-4048-b7d8-3c5699caadcc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:39:29.794621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.794545    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7ff18c8-3caf-4048-b7d8-3c5699caadcc" (UID: "d7ff18c8-3caf-4048-b7d8-3c5699caadcc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:39:29.794621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.794559    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istiod-ca-cert\") pod \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") "
Sep 08 15:39:29.795104 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.794627    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-workload-certs\") pod \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\" (UID: \"d7ff18c8-3caf-4048-b7d8-3c5699caadcc\") "
Sep 08 15:39:29.795224 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.795195    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7ff18c8-3caf-4048-b7d8-3c5699caadcc" (UID: "d7ff18c8-3caf-4048-b7d8-3c5699caadcc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:39:29.795664 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.795532    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7ff18c8-3caf-4048-b7d8-3c5699caadcc" (UID: "d7ff18c8-3caf-4048-b7d8-3c5699caadcc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:39:29.795664 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.795589    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:29.795664 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.795618    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:29.795664 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.795634    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:29.798197 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.794723    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7ff18c8-3caf-4048-b7d8-3c5699caadcc" (UID: "d7ff18c8-3caf-4048-b7d8-3c5699caadcc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:39:29.799160 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.798857    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7ff18c8-3caf-4048-b7d8-3c5699caadcc" (UID: "d7ff18c8-3caf-4048-b7d8-3c5699caadcc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:39:29.799160 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.799120    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7ff18c8-3caf-4048-b7d8-3c5699caadcc" (UID: "d7ff18c8-3caf-4048-b7d8-3c5699caadcc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:39:29.801758 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.801723    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-kube-api-access-fhr2m" (OuterVolumeSpecName: "kube-api-access-fhr2m") pod "d7ff18c8-3caf-4048-b7d8-3c5699caadcc" (UID: "d7ff18c8-3caf-4048-b7d8-3c5699caadcc"). InnerVolumeSpecName "kube-api-access-fhr2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:39:29.806329 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.806302    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7ff18c8-3caf-4048-b7d8-3c5699caadcc" (UID: "d7ff18c8-3caf-4048-b7d8-3c5699caadcc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:39:29.896229 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.896193    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:29.896401 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.896253    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhr2m\" (UniqueName: \"kubernetes.io/projected/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-kube-api-access-fhr2m\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:29.896401 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.896269    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:29.896401 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.896281    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:29.896401 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.896292    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:29.896401 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:29.896303    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ff18c8-3caf-4048-b7d8-3c5699caadcc-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:30.279467 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.279374    2575 generic.go:356] "Generic (PLEG): container finished" podID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerID="d8f4f4531382642356626409f3b332411f61c0ef02a721bfa45775091ef9c723" exitCode=0
Sep 08 15:39:30.279467 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.279442    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"
Sep 08 15:39:30.279727 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.279462    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" event={"ID":"d7ff18c8-3caf-4048-b7d8-3c5699caadcc","Type":"ContainerDied","Data":"d8f4f4531382642356626409f3b332411f61c0ef02a721bfa45775091ef9c723"}
Sep 08 15:39:30.279727 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.279586    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2" event={"ID":"d7ff18c8-3caf-4048-b7d8-3c5699caadcc","Type":"ContainerDied","Data":"b1ec0fd636118129058803b8ee340cf35760ae874381ebdc2357e4a21afc6ee2"}
Sep 08 15:39:30.279727 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.279621    2575 scope.go:117] "RemoveContainer" containerID="d8f4f4531382642356626409f3b332411f61c0ef02a721bfa45775091ef9c723"
Sep 08 15:39:30.281306 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.281281    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" event={"ID":"5bd6d10e-779b-4f2f-94bd-5333cc0093b0","Type":"ContainerStarted","Data":"c2856de54c44994430ff3e432146b058d2d95c8fff71168a6f4edcc628c0ef6c"}
Sep 08 15:39:30.281391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.281335    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" event={"ID":"5bd6d10e-779b-4f2f-94bd-5333cc0093b0","Type":"ContainerStarted","Data":"c824d19f6ea4542fd2859671422b75efe6f5968ae1ba08f9a5aad28d63a601ac"}
Sep 08 15:39:30.289022 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.289004    2575 scope.go:117] "RemoveContainer" containerID="e8170b190ec41340512f9477d78b52f884de6e8ed7d1abfd112b20fd4e760c39"
Sep 08 15:39:30.310839 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.310824    2575 scope.go:117] "RemoveContainer" containerID="d8f4f4531382642356626409f3b332411f61c0ef02a721bfa45775091ef9c723"
Sep 08 15:39:30.311106 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:39:30.311082    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8f4f4531382642356626409f3b332411f61c0ef02a721bfa45775091ef9c723\": container with ID starting with d8f4f4531382642356626409f3b332411f61c0ef02a721bfa45775091ef9c723 not found: ID does not exist" containerID="d8f4f4531382642356626409f3b332411f61c0ef02a721bfa45775091ef9c723"
Sep 08 15:39:30.311178 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.311108    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8f4f4531382642356626409f3b332411f61c0ef02a721bfa45775091ef9c723"} err="failed to get container status \"d8f4f4531382642356626409f3b332411f61c0ef02a721bfa45775091ef9c723\": rpc error: code = NotFound desc = could not find container \"d8f4f4531382642356626409f3b332411f61c0ef02a721bfa45775091ef9c723\": container with ID starting with d8f4f4531382642356626409f3b332411f61c0ef02a721bfa45775091ef9c723 not found: ID does not exist"
Sep 08 15:39:30.311178 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.311123    2575 scope.go:117] "RemoveContainer" containerID="e8170b190ec41340512f9477d78b52f884de6e8ed7d1abfd112b20fd4e760c39"
Sep 08 15:39:30.311384 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:39:30.311363    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8170b190ec41340512f9477d78b52f884de6e8ed7d1abfd112b20fd4e760c39\": container with ID starting with e8170b190ec41340512f9477d78b52f884de6e8ed7d1abfd112b20fd4e760c39 not found: ID does not exist" containerID="e8170b190ec41340512f9477d78b52f884de6e8ed7d1abfd112b20fd4e760c39"
Sep 08 15:39:30.311453 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.311393    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8170b190ec41340512f9477d78b52f884de6e8ed7d1abfd112b20fd4e760c39"} err="failed to get container status \"e8170b190ec41340512f9477d78b52f884de6e8ed7d1abfd112b20fd4e760c39\": rpc error: code = NotFound desc = could not find container \"e8170b190ec41340512f9477d78b52f884de6e8ed7d1abfd112b20fd4e760c39\": container with ID starting with e8170b190ec41340512f9477d78b52f884de6e8ed7d1abfd112b20fd4e760c39 not found: ID does not exist"
Sep 08 15:39:30.315038 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.314995    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" podStartSLOduration=1.314983866 podStartE2EDuration="1.314983866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:39:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:39:30.313520938 +0000 UTC m=+1818.399470973" watchObservedRunningTime="2026-09-08 15:39:30.314983866 +0000 UTC m=+1818.400933907"
Sep 08 15:39:30.336437 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.336416    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"]
Sep 08 15:39:30.339778 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.339759    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-149fdb-openshift-default-7bcd95755d-87ww2"]
Sep 08 15:39:30.348765 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.348740    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:30.353750 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.353730    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:30.433688 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:30.433646    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" path="/var/lib/kubelet/pods/d7ff18c8-3caf-4048-b7d8-3c5699caadcc/volumes"
Sep 08 15:39:31.287559 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:31.287527    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:31.288414 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:31.288394    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:32.417106 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:32.417069    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:32.514011 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:32.513975    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:33.497430 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.497397    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"]
Sep 08 15:39:33.497897 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.497881    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerName="istio-proxy"
Sep 08 15:39:33.497943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.497899    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerName="istio-proxy"
Sep 08 15:39:33.497943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.497912    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerName="istio-proxy"
Sep 08 15:39:33.497943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.497917    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerName="istio-proxy"
Sep 08 15:39:33.497943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.497923    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerName="istio-proxy"
Sep 08 15:39:33.497943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.497930    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b28902e-c61b-437e-92bb-94e8bb7f9d94" containerName="istio-proxy"
Sep 08 15:39:33.498094 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.498008    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerName="istio-proxy"
Sep 08 15:39:33.502831 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.502807    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:33.505712 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.505692    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-d97358-serving-cert\""
Sep 08 15:39:33.517408 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.517382    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"]
Sep 08 15:39:33.581059 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.581024    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:33.635579 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.635537    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:33.635579 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.635577    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:33.635789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.635650    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxnvb\" (UniqueName: \"kubernetes.io/projected/1720dbba-3a6d-4253-819d-193e50c6053b-kube-api-access-dxnvb\") pod \"maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:33.635789 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.635750    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1720dbba-3a6d-4253-819d-193e50c6053b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:33.661765 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.661720    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:33.736943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.736908    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1720dbba-3a6d-4253-819d-193e50c6053b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:33.737103 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.737012    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:33.737103 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.737040    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:33.737103 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.737091    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxnvb\" (UniqueName: \"kubernetes.io/projected/1720dbba-3a6d-4253-819d-193e50c6053b-kube-api-access-dxnvb\") pod \"maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:33.737205 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:39:33.737156    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-serving-cert: secret "maas-api-e2e-ipp-cleanup-d97358-serving-cert" not found
Sep 08 15:39:33.737239 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:39:33.737220    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-maas-api-tls podName:1720dbba-3a6d-4253-819d-193e50c6053b nodeName:}" failed. No retries permitted until 2026-09-08 15:39:34.237202655 +0000 UTC m=+1822.323152674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4" (UID: "1720dbba-3a6d-4253-819d-193e50c6053b") : secret "maas-api-e2e-ipp-cleanup-d97358-serving-cert" not found
Sep 08 15:39:33.737706 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.737684    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1720dbba-3a6d-4253-819d-193e50c6053b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:33.739628 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.739605    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:33.746329 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.746306    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxnvb\" (UniqueName: \"kubernetes.io/projected/1720dbba-3a6d-4253-819d-193e50c6053b-kube-api-access-dxnvb\") pod \"maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:33.952102 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:33.952067    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:34.227103 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:34.227013    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:34.242261 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:34.242219    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:34.244926 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:34.244898    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:34.327668 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:34.327625    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:34.415285 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:34.415252    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:34.698891 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:34.698864    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"]
Sep 08 15:39:34.701449 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:39:34.701424    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1720dbba_3a6d_4253_819d_193e50c6053b.slice/crio-1864991f80d39b94f69feee234deb0cad77dbbb75caf6b0e3f6c9441a7755d3f WatchSource:0}: Error finding container 1864991f80d39b94f69feee234deb0cad77dbbb75caf6b0e3f6c9441a7755d3f: Status 404 returned error can't find the container with id 1864991f80d39b94f69feee234deb0cad77dbbb75caf6b0e3f6c9441a7755d3f
Sep 08 15:39:35.101567 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.101470    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"]
Sep 08 15:39:35.101730 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.101693    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc" podUID="50535ed2-f951-4199-8762-278e26fa046e" containerName="maas-api" containerID="cri-o://f197645b36c859850a9041178c783c53e5062f96923875aa7745d44daf8f8c6d" gracePeriod=30
Sep 08 15:39:35.142240 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.142208    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:35.244156 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.244120    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:35.310725 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.310690    2575 generic.go:356] "Generic (PLEG): container finished" podID="50535ed2-f951-4199-8762-278e26fa046e" containerID="f197645b36c859850a9041178c783c53e5062f96923875aa7745d44daf8f8c6d" exitCode=0
Sep 08 15:39:35.310880 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.310758    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc" event={"ID":"50535ed2-f951-4199-8762-278e26fa046e","Type":"ContainerDied","Data":"f197645b36c859850a9041178c783c53e5062f96923875aa7745d44daf8f8c6d"}
Sep 08 15:39:35.312200 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.312176    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4" event={"ID":"1720dbba-3a6d-4253-819d-193e50c6053b","Type":"ContainerStarted","Data":"1864991f80d39b94f69feee234deb0cad77dbbb75caf6b0e3f6c9441a7755d3f"}
Sep 08 15:39:35.331919 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.331891    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:35.354357 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.354332    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:39:35.454030 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.454010    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-metrics-certs\") pod \"50535ed2-f951-4199-8762-278e26fa046e\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") "
Sep 08 15:39:35.454152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.454088    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50535ed2-f951-4199-8762-278e26fa046e-openshift-service-ca\") pod \"50535ed2-f951-4199-8762-278e26fa046e\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") "
Sep 08 15:39:35.454152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.454139    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29bnz\" (UniqueName: \"kubernetes.io/projected/50535ed2-f951-4199-8762-278e26fa046e-kube-api-access-29bnz\") pod \"50535ed2-f951-4199-8762-278e26fa046e\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") "
Sep 08 15:39:35.454224 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.454189    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-maas-api-tls\") pod \"50535ed2-f951-4199-8762-278e26fa046e\" (UID: \"50535ed2-f951-4199-8762-278e26fa046e\") "
Sep 08 15:39:35.454540 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.454479    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50535ed2-f951-4199-8762-278e26fa046e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "50535ed2-f951-4199-8762-278e26fa046e" (UID: "50535ed2-f951-4199-8762-278e26fa046e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:39:35.456039 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.456008    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "50535ed2-f951-4199-8762-278e26fa046e" (UID: "50535ed2-f951-4199-8762-278e26fa046e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:39:35.456170 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.456151    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "50535ed2-f951-4199-8762-278e26fa046e" (UID: "50535ed2-f951-4199-8762-278e26fa046e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:39:35.456170 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.456156    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50535ed2-f951-4199-8762-278e26fa046e-kube-api-access-29bnz" (OuterVolumeSpecName: "kube-api-access-29bnz") pod "50535ed2-f951-4199-8762-278e26fa046e" (UID: "50535ed2-f951-4199-8762-278e26fa046e"). InnerVolumeSpecName "kube-api-access-29bnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:39:35.555486 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.555450    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50535ed2-f951-4199-8762-278e26fa046e-openshift-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:35.555486 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.555481    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29bnz\" (UniqueName: \"kubernetes.io/projected/50535ed2-f951-4199-8762-278e26fa046e-kube-api-access-29bnz\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:35.555694 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.555519    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-maas-api-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:35.555694 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:35.555531    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50535ed2-f951-4199-8762-278e26fa046e-metrics-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:39:36.319273 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:36.319245    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"
Sep 08 15:39:36.319735 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:36.319277    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc" event={"ID":"50535ed2-f951-4199-8762-278e26fa046e","Type":"ContainerDied","Data":"5cac3addadf742bf7c78ed7f2fa8bbb72ae675bb132482391e16642f8d4b94b1"}
Sep 08 15:39:36.319735 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:36.319342    2575 scope.go:117] "RemoveContainer" containerID="f197645b36c859850a9041178c783c53e5062f96923875aa7745d44daf8f8c6d"
Sep 08 15:39:36.321096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:36.321063    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4" event={"ID":"1720dbba-3a6d-4253-819d-193e50c6053b","Type":"ContainerStarted","Data":"004f451e8628eb403f6a2aac9772c35f0c01812f77039f0c2bf9cb3139db3c54"}
Sep 08 15:39:36.321202 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:36.321131    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:36.348101 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:36.348053    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4" podStartSLOduration=2.643483135 podStartE2EDuration="3.348036264s" totalImagesPullingTime="704.553129ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:39:36.346170985 +0000 UTC m=+1824.432121026" watchObservedRunningTime="2026-09-08 15:39:36.348036264 +0000 UTC m=+1824.433986308"
Sep 08 15:39:36.361481 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:36.361427    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"]
Sep 08 15:39:36.363351 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:36.363327    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-04285b-64fb94c4d-h4pmc"]
Sep 08 15:39:36.431997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:36.431969    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50535ed2-f951-4199-8762-278e26fa046e" path="/var/lib/kubelet/pods/50535ed2-f951-4199-8762-278e26fa046e/volumes"
Sep 08 15:39:37.342459 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:37.342425    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:37.468166 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:37.468118    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:41.862864 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:41.862822    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:42.333265 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:42.333182    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:39:43.222542 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:43.222481    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:43.841760 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:43.841718    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:39:54.428403 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:54.428370    2575 generic.go:356] "Generic (PLEG): container finished" podID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerID="c2856de54c44994430ff3e432146b058d2d95c8fff71168a6f4edcc628c0ef6c" exitCode=0
Sep 08 15:39:54.431389 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:54.431364    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" event={"ID":"5bd6d10e-779b-4f2f-94bd-5333cc0093b0","Type":"ContainerDied","Data":"c2856de54c44994430ff3e432146b058d2d95c8fff71168a6f4edcc628c0ef6c"}
Sep 08 15:39:54.431768 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:54.431752    2575 scope.go:117] "RemoveContainer" containerID="c2856de54c44994430ff3e432146b058d2d95c8fff71168a6f4edcc628c0ef6c"
Sep 08 15:39:54.432865 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:54.432837    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:39:54.432926 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:54.432908    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:39:54.432965 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:54.432939    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:39:55.435151 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:55.435121    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" event={"ID":"5bd6d10e-779b-4f2f-94bd-5333cc0093b0","Type":"ContainerStarted","Data":"43282452c742353e15faaa14db45bffdbeeaec45a9b88ba4806dc7a55aa025fb"}
Sep 08 15:39:56.349405 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:56.349367    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:39:56.349652 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:56.349625    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 08 15:39:56.349737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:56.349692    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 08 15:39:57.299868 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:57.299817    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:39:57.349113 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:57.349073    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 08 15:39:57.349260 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:57.349123    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 08 15:39:58.349209 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:58.349172    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 08 15:39:58.349654 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:58.349239    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 08 15:39:59.349509 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:59.349449    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 08 15:39:59.349885 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:39:59.349536    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 08 15:40:00.348966 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:00.348930    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 08 15:40:00.349149 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:00.348982    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 08 15:40:01.288421 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:01.288388    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:40:01.349650 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:01.349616    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 08 15:40:01.349812 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:01.349676    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 08 15:40:02.352337 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:02.352307    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:40:02.353163 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:02.353145    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:40:04.204388 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.204354    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"]
Sep 08 15:40:04.204879 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.204606    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4" podUID="1720dbba-3a6d-4253-819d-193e50c6053b" containerName="maas-api" containerID="cri-o://004f451e8628eb403f6a2aac9772c35f0c01812f77039f0c2bf9cb3139db3c54" gracePeriod=30
Sep 08 15:40:04.239617 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.239587    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:04.347588 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.347553    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:04.443336 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.443263    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:04.456773 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.456705    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:40:04.473993 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.473966    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-maas-api-tls\") pod \"1720dbba-3a6d-4253-819d-193e50c6053b\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") "
Sep 08 15:40:04.474170 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.474045    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1720dbba-3a6d-4253-819d-193e50c6053b-openshift-service-ca\") pod \"1720dbba-3a6d-4253-819d-193e50c6053b\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") "
Sep 08 15:40:04.474170 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.474138    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-metrics-certs\") pod \"1720dbba-3a6d-4253-819d-193e50c6053b\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") "
Sep 08 15:40:04.474170 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.474164    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxnvb\" (UniqueName: \"kubernetes.io/projected/1720dbba-3a6d-4253-819d-193e50c6053b-kube-api-access-dxnvb\") pod \"1720dbba-3a6d-4253-819d-193e50c6053b\" (UID: \"1720dbba-3a6d-4253-819d-193e50c6053b\") "
Sep 08 15:40:04.474438 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.474412    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1720dbba-3a6d-4253-819d-193e50c6053b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1720dbba-3a6d-4253-819d-193e50c6053b" (UID: "1720dbba-3a6d-4253-819d-193e50c6053b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:40:04.474582 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.474565    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1720dbba-3a6d-4253-819d-193e50c6053b-openshift-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:04.476633 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.476603    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1720dbba-3a6d-4253-819d-193e50c6053b" (UID: "1720dbba-3a6d-4253-819d-193e50c6053b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:40:04.476856 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.476827    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1720dbba-3a6d-4253-819d-193e50c6053b" (UID: "1720dbba-3a6d-4253-819d-193e50c6053b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:40:04.477147 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.477122    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1720dbba-3a6d-4253-819d-193e50c6053b-kube-api-access-dxnvb" (OuterVolumeSpecName: "kube-api-access-dxnvb") pod "1720dbba-3a6d-4253-819d-193e50c6053b" (UID: "1720dbba-3a6d-4253-819d-193e50c6053b"). InnerVolumeSpecName "kube-api-access-dxnvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:40:04.485282 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.485084    2575 generic.go:356] "Generic (PLEG): container finished" podID="1720dbba-3a6d-4253-819d-193e50c6053b" containerID="004f451e8628eb403f6a2aac9772c35f0c01812f77039f0c2bf9cb3139db3c54" exitCode=0
Sep 08 15:40:04.485282 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.485126    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"
Sep 08 15:40:04.485282 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.485129    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4" event={"ID":"1720dbba-3a6d-4253-819d-193e50c6053b","Type":"ContainerDied","Data":"004f451e8628eb403f6a2aac9772c35f0c01812f77039f0c2bf9cb3139db3c54"}
Sep 08 15:40:04.485282 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.485247    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4" event={"ID":"1720dbba-3a6d-4253-819d-193e50c6053b","Type":"ContainerDied","Data":"1864991f80d39b94f69feee234deb0cad77dbbb75caf6b0e3f6c9441a7755d3f"}
Sep 08 15:40:04.485282 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.485281    2575 scope.go:117] "RemoveContainer" containerID="004f451e8628eb403f6a2aac9772c35f0c01812f77039f0c2bf9cb3139db3c54"
Sep 08 15:40:04.503065 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.503041    2575 scope.go:117] "RemoveContainer" containerID="004f451e8628eb403f6a2aac9772c35f0c01812f77039f0c2bf9cb3139db3c54"
Sep 08 15:40:04.503377 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:40:04.503350    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"004f451e8628eb403f6a2aac9772c35f0c01812f77039f0c2bf9cb3139db3c54\": container with ID starting with 004f451e8628eb403f6a2aac9772c35f0c01812f77039f0c2bf9cb3139db3c54 not found: ID does not exist" containerID="004f451e8628eb403f6a2aac9772c35f0c01812f77039f0c2bf9cb3139db3c54"
Sep 08 15:40:04.503464 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.503384    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"004f451e8628eb403f6a2aac9772c35f0c01812f77039f0c2bf9cb3139db3c54"} err="failed to get container status \"004f451e8628eb403f6a2aac9772c35f0c01812f77039f0c2bf9cb3139db3c54\": rpc error: code = NotFound desc = could not find container \"004f451e8628eb403f6a2aac9772c35f0c01812f77039f0c2bf9cb3139db3c54\": container with ID starting with 004f451e8628eb403f6a2aac9772c35f0c01812f77039f0c2bf9cb3139db3c54 not found: ID does not exist"
Sep 08 15:40:04.515559 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.515528    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"]
Sep 08 15:40:04.522993 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.522973    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d97358-588579b8bf-7j7j4"]
Sep 08 15:40:04.528581 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.528535    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:04.575311 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.575288    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-metrics-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:04.575311 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.575310    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxnvb\" (UniqueName: \"kubernetes.io/projected/1720dbba-3a6d-4253-819d-193e50c6053b-kube-api-access-dxnvb\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:04.575441 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.575320    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1720dbba-3a6d-4253-819d-193e50c6053b-maas-api-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:04.860741 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:04.860710    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:06.437021 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:06.436979    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1720dbba-3a6d-4253-819d-193e50c6053b" path="/var/lib/kubelet/pods/1720dbba-3a6d-4253-819d-193e50c6053b/volumes"
Sep 08 15:40:11.170943 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:11.170900    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:11.270122 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:11.270083    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:11.423413 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:11.423331    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"]
Sep 08 15:40:11.423641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:11.423595    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerName="istio-proxy" containerID="cri-o://43282452c742353e15faaa14db45bffdbeeaec45a9b88ba4806dc7a55aa025fb" gracePeriod=30
Sep 08 15:40:11.456434 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:11.456405    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:11.645818 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:11.645780    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:40:15.117520 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:15.117474    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:16.552155 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.552102    2575 generic.go:356] "Generic (PLEG): container finished" podID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerID="43282452c742353e15faaa14db45bffdbeeaec45a9b88ba4806dc7a55aa025fb" exitCode=0
Sep 08 15:40:16.552474 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.552170    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" event={"ID":"5bd6d10e-779b-4f2f-94bd-5333cc0093b0","Type":"ContainerDied","Data":"43282452c742353e15faaa14db45bffdbeeaec45a9b88ba4806dc7a55aa025fb"}
Sep 08 15:40:16.552474 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.552239    2575 scope.go:117] "RemoveContainer" containerID="c2856de54c44994430ff3e432146b058d2d95c8fff71168a6f4edcc628c0ef6c"
Sep 08 15:40:16.661397 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.661373    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:40:16.796481 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.796404    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-workload-certs\") pod \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") "
Sep 08 15:40:16.796481 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.796441    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rzgl\" (UniqueName: \"kubernetes.io/projected/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-kube-api-access-2rzgl\") pod \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") "
Sep 08 15:40:16.796481 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.796474    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-data\") pod \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") "
Sep 08 15:40:16.796740 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.796611    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-credential-socket\") pod \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") "
Sep 08 15:40:16.796740 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.796667    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-podinfo\") pod \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") "
Sep 08 15:40:16.796740 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.796712    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-data" (OuterVolumeSpecName: "istio-data") pod "5bd6d10e-779b-4f2f-94bd-5333cc0093b0" (UID: "5bd6d10e-779b-4f2f-94bd-5333cc0093b0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:40:16.796740 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.796725    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-envoy\") pod \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") "
Sep 08 15:40:16.796940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.796755    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5bd6d10e-779b-4f2f-94bd-5333cc0093b0" (UID: "5bd6d10e-779b-4f2f-94bd-5333cc0093b0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:40:16.796940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.796779    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-workload-socket\") pod \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") "
Sep 08 15:40:16.796940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.796809    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-token\") pod \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") "
Sep 08 15:40:16.796940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.796834    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istiod-ca-cert\") pod \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\" (UID: \"5bd6d10e-779b-4f2f-94bd-5333cc0093b0\") "
Sep 08 15:40:16.796940 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.796860    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5bd6d10e-779b-4f2f-94bd-5333cc0093b0" (UID: "5bd6d10e-779b-4f2f-94bd-5333cc0093b0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:40:16.797197 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.797040    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5bd6d10e-779b-4f2f-94bd-5333cc0093b0" (UID: "5bd6d10e-779b-4f2f-94bd-5333cc0093b0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:40:16.797197 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.797163    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:16.797197 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.797180    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:16.797197 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.797191    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:16.797197 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.797200    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:16.797396 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.797209    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5bd6d10e-779b-4f2f-94bd-5333cc0093b0" (UID: "5bd6d10e-779b-4f2f-94bd-5333cc0093b0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:40:16.799076 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.799048    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-kube-api-access-2rzgl" (OuterVolumeSpecName: "kube-api-access-2rzgl") pod "5bd6d10e-779b-4f2f-94bd-5333cc0093b0" (UID: "5bd6d10e-779b-4f2f-94bd-5333cc0093b0"). InnerVolumeSpecName "kube-api-access-2rzgl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:40:16.799076 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.799045    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5bd6d10e-779b-4f2f-94bd-5333cc0093b0" (UID: "5bd6d10e-779b-4f2f-94bd-5333cc0093b0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:40:16.799197 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.799122    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-token" (OuterVolumeSpecName: "istio-token") pod "5bd6d10e-779b-4f2f-94bd-5333cc0093b0" (UID: "5bd6d10e-779b-4f2f-94bd-5333cc0093b0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:40:16.799197 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.799139    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5bd6d10e-779b-4f2f-94bd-5333cc0093b0" (UID: "5bd6d10e-779b-4f2f-94bd-5333cc0093b0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:40:16.897821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.897789    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:16.897821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.897818    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:16.897821 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.897827    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:16.898037 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.897834    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:16.898037 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:16.897842    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rzgl\" (UniqueName: \"kubernetes.io/projected/5bd6d10e-779b-4f2f-94bd-5333cc0093b0-kube-api-access-2rzgl\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:17.559233 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:17.559203    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"
Sep 08 15:40:17.559661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:17.559233    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv" event={"ID":"5bd6d10e-779b-4f2f-94bd-5333cc0093b0","Type":"ContainerDied","Data":"c824d19f6ea4542fd2859671422b75efe6f5968ae1ba08f9a5aad28d63a601ac"}
Sep 08 15:40:17.559661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:17.559283    2575 scope.go:117] "RemoveContainer" containerID="43282452c742353e15faaa14db45bffdbeeaec45a9b88ba4806dc7a55aa025fb"
Sep 08 15:40:17.582911 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:17.582887    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"]
Sep 08 15:40:17.590693 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:17.590670    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d97358-openshift-default-5bbfff9db8-6qbzv"]
Sep 08 15:40:18.431724 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:18.431680    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" path="/var/lib/kubelet/pods/5bd6d10e-779b-4f2f-94bd-5333cc0093b0/volumes"
Sep 08 15:40:30.245128 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:30.245092    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:40:30.736956 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:30.736902    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:30.844081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:30.844040    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:30.942552 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:30.942483    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:31.052998 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:31.052912    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:31.463849 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:31.463817    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:36.402743 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:36.402666    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"]
Sep 08 15:40:36.496765 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:36.496730    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:39.644199 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:39.644168    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:40:43.012961 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:43.012928    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:43.114225 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:43.114191    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:40:51.422483 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.422433    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk" podUID="a348d026-4378-47a3-a836-97d173458c5b" containerName="main" containerID="cri-o://b574a020a69abe8cb49866e9252f6e1400d683ae157b453feeffce92db3be7aa" gracePeriod=45
Sep 08 15:40:51.664286 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.664263    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:40:51.740079 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.739985    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gqdz\" (UniqueName: \"kubernetes.io/projected/a348d026-4378-47a3-a836-97d173458c5b-kube-api-access-9gqdz\") pod \"a348d026-4378-47a3-a836-97d173458c5b\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") "
Sep 08 15:40:51.740079 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.740049    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-tmp-dir\") pod \"a348d026-4378-47a3-a836-97d173458c5b\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") "
Sep 08 15:40:51.740321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.740086    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a348d026-4378-47a3-a836-97d173458c5b-tls-certs\") pod \"a348d026-4378-47a3-a836-97d173458c5b\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") "
Sep 08 15:40:51.740321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.740121    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-model-cache\") pod \"a348d026-4378-47a3-a836-97d173458c5b\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") "
Sep 08 15:40:51.740321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.740198    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-dshm\") pod \"a348d026-4378-47a3-a836-97d173458c5b\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") "
Sep 08 15:40:51.740321 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.740254    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-home\") pod \"a348d026-4378-47a3-a836-97d173458c5b\" (UID: \"a348d026-4378-47a3-a836-97d173458c5b\") "
Sep 08 15:40:51.740597 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.740393    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a348d026-4378-47a3-a836-97d173458c5b" (UID: "a348d026-4378-47a3-a836-97d173458c5b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:40:51.740597 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.740480    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-model-cache" (OuterVolumeSpecName: "model-cache") pod "a348d026-4378-47a3-a836-97d173458c5b" (UID: "a348d026-4378-47a3-a836-97d173458c5b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:40:51.740716 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.740643    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-home" (OuterVolumeSpecName: "home") pod "a348d026-4378-47a3-a836-97d173458c5b" (UID: "a348d026-4378-47a3-a836-97d173458c5b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:40:51.740776 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.740730    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:51.740776 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.740753    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:51.742468 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.742405    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a348d026-4378-47a3-a836-97d173458c5b-kube-api-access-9gqdz" (OuterVolumeSpecName: "kube-api-access-9gqdz") pod "a348d026-4378-47a3-a836-97d173458c5b" (UID: "a348d026-4378-47a3-a836-97d173458c5b"). InnerVolumeSpecName "kube-api-access-9gqdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:40:51.742606 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.742478    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a348d026-4378-47a3-a836-97d173458c5b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a348d026-4378-47a3-a836-97d173458c5b" (UID: "a348d026-4378-47a3-a836-97d173458c5b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:40:51.742606 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.742530    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-dshm" (OuterVolumeSpecName: "dshm") pod "a348d026-4378-47a3-a836-97d173458c5b" (UID: "a348d026-4378-47a3-a836-97d173458c5b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:40:51.745869 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.745844    2575 generic.go:356] "Generic (PLEG): container finished" podID="a348d026-4378-47a3-a836-97d173458c5b" containerID="b574a020a69abe8cb49866e9252f6e1400d683ae157b453feeffce92db3be7aa" exitCode=0
Sep 08 15:40:51.745978 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.745888    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"
Sep 08 15:40:51.745978 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.745926    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk" event={"ID":"a348d026-4378-47a3-a836-97d173458c5b","Type":"ContainerDied","Data":"b574a020a69abe8cb49866e9252f6e1400d683ae157b453feeffce92db3be7aa"}
Sep 08 15:40:51.745978 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.745968    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk" event={"ID":"a348d026-4378-47a3-a836-97d173458c5b","Type":"ContainerDied","Data":"b5af01f70d4918bd3222856b835753da481125e751266daa12f730a50fe68533"}
Sep 08 15:40:51.746106 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.745985    2575 scope.go:117] "RemoveContainer" containerID="b574a020a69abe8cb49866e9252f6e1400d683ae157b453feeffce92db3be7aa"
Sep 08 15:40:51.762648 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.762630    2575 scope.go:117] "RemoveContainer" containerID="b574a020a69abe8cb49866e9252f6e1400d683ae157b453feeffce92db3be7aa"
Sep 08 15:40:51.762908 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:40:51.762887    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b574a020a69abe8cb49866e9252f6e1400d683ae157b453feeffce92db3be7aa\": container with ID starting with b574a020a69abe8cb49866e9252f6e1400d683ae157b453feeffce92db3be7aa not found: ID does not exist" containerID="b574a020a69abe8cb49866e9252f6e1400d683ae157b453feeffce92db3be7aa"
Sep 08 15:40:51.762955 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.762917    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b574a020a69abe8cb49866e9252f6e1400d683ae157b453feeffce92db3be7aa"} err="failed to get container status \"b574a020a69abe8cb49866e9252f6e1400d683ae157b453feeffce92db3be7aa\": rpc error: code = NotFound desc = could not find container \"b574a020a69abe8cb49866e9252f6e1400d683ae157b453feeffce92db3be7aa\": container with ID starting with b574a020a69abe8cb49866e9252f6e1400d683ae157b453feeffce92db3be7aa not found: ID does not exist"
Sep 08 15:40:51.774007 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.773984    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"]
Sep 08 15:40:51.778707 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.778681    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-4a2c80/ipp-route-4a2c80-kserve-89b558f87-vd9pk"]
Sep 08 15:40:51.841681 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.841656    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gqdz\" (UniqueName: \"kubernetes.io/projected/a348d026-4378-47a3-a836-97d173458c5b-kube-api-access-9gqdz\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:51.841794 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.841681    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a348d026-4378-47a3-a836-97d173458c5b-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:51.841794 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.841693    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:51.841794 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:51.841701    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a348d026-4378-47a3-a836-97d173458c5b-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:40:52.432025 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:40:52.431993    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a348d026-4378-47a3-a836-97d173458c5b" path="/var/lib/kubelet/pods/a348d026-4378-47a3-a836-97d173458c5b/volumes"
Sep 08 15:41:12.976595 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:12.976558    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:14.051965 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:14.051930    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:20.293601 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:20.293557    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:24.139318 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:24.139267    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:35.302995 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:35.302933    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:35.370997 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:35.370962    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:35.571721 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:35.571640    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"]
Sep 08 15:41:35.571917 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:35.571890    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy" containerID="cri-o://cabbbc573c0a4cc01a9cdea421ed70be8e9d21f0cb9cf0addcdd564770b13c8f" gracePeriod=30
Sep 08 15:41:35.597436 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:35.597409    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:36.154565 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:36.154532    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:40.821640 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.821616    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:41:40.973400 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.973372    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-workload-socket\") pod \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") "
Sep 08 15:41:40.973571 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.973415    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-token\") pod \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") "
Sep 08 15:41:40.973571 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.973522    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istiod-ca-cert\") pod \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") "
Sep 08 15:41:40.973571 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.973554    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-podinfo\") pod \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") "
Sep 08 15:41:40.973743 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.973577    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-workload-certs\") pod \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") "
Sep 08 15:41:40.973743 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.973657    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" (UID: "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:41:40.973743 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.973722    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dldw8\" (UniqueName: \"kubernetes.io/projected/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-kube-api-access-dldw8\") pod \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") "
Sep 08 15:41:40.973903 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.973754    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-envoy\") pod \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") "
Sep 08 15:41:40.973903 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.973789    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-data\") pod \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") "
Sep 08 15:41:40.973903 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.973812    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-credential-socket\") pod \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\" (UID: \"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6\") "
Sep 08 15:41:40.973903 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.973849    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" (UID: "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:41:40.974107 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.973960    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" (UID: "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:41:40.974107 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.974089    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-data" (OuterVolumeSpecName: "istio-data") pod "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" (UID: "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:41:40.974305 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.974265    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" (UID: "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:41:40.974305 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.974269    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:40.974516 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.974315    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:40.974516 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.974326    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:40.974516 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.974335    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:40.975932 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.975906    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-kube-api-access-dldw8" (OuterVolumeSpecName: "kube-api-access-dldw8") pod "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" (UID: "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6"). InnerVolumeSpecName "kube-api-access-dldw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:41:40.976016 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.975949    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" (UID: "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:41:40.976145 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.976114    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-token" (OuterVolumeSpecName: "istio-token") pod "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" (UID: "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:41:40.976256 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:40.976178    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" (UID: "b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:41:41.010415 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.010386    2575 generic.go:356] "Generic (PLEG): container finished" podID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerID="cabbbc573c0a4cc01a9cdea421ed70be8e9d21f0cb9cf0addcdd564770b13c8f" exitCode=0
Sep 08 15:41:41.010525 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.010434    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"
Sep 08 15:41:41.010525 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.010464    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" event={"ID":"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6","Type":"ContainerDied","Data":"cabbbc573c0a4cc01a9cdea421ed70be8e9d21f0cb9cf0addcdd564770b13c8f"}
Sep 08 15:41:41.010609 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.010544    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9" event={"ID":"b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6","Type":"ContainerDied","Data":"7d7ed6ea917a4f1cd67e7f86df615dbc1a4c20ed23fae1e67cb6941014c885e4"}
Sep 08 15:41:41.010609 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.010569    2575 scope.go:117] "RemoveContainer" containerID="cabbbc573c0a4cc01a9cdea421ed70be8e9d21f0cb9cf0addcdd564770b13c8f"
Sep 08 15:41:41.019580 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.019559    2575 scope.go:117] "RemoveContainer" containerID="a359a057666ff7c0814a65b10713f37f11896b3d384aeaac26c679d17ceb9fe4"
Sep 08 15:41:41.027535 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.027512    2575 scope.go:117] "RemoveContainer" containerID="cabbbc573c0a4cc01a9cdea421ed70be8e9d21f0cb9cf0addcdd564770b13c8f"
Sep 08 15:41:41.027800 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:41:41.027783    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cabbbc573c0a4cc01a9cdea421ed70be8e9d21f0cb9cf0addcdd564770b13c8f\": container with ID starting with cabbbc573c0a4cc01a9cdea421ed70be8e9d21f0cb9cf0addcdd564770b13c8f not found: ID does not exist" containerID="cabbbc573c0a4cc01a9cdea421ed70be8e9d21f0cb9cf0addcdd564770b13c8f"
Sep 08 15:41:41.027880 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.027810    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cabbbc573c0a4cc01a9cdea421ed70be8e9d21f0cb9cf0addcdd564770b13c8f"} err="failed to get container status \"cabbbc573c0a4cc01a9cdea421ed70be8e9d21f0cb9cf0addcdd564770b13c8f\": rpc error: code = NotFound desc = could not find container \"cabbbc573c0a4cc01a9cdea421ed70be8e9d21f0cb9cf0addcdd564770b13c8f\": container with ID starting with cabbbc573c0a4cc01a9cdea421ed70be8e9d21f0cb9cf0addcdd564770b13c8f not found: ID does not exist"
Sep 08 15:41:41.027880 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.027832    2575 scope.go:117] "RemoveContainer" containerID="a359a057666ff7c0814a65b10713f37f11896b3d384aeaac26c679d17ceb9fe4"
Sep 08 15:41:41.028089 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:41:41.028072    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a359a057666ff7c0814a65b10713f37f11896b3d384aeaac26c679d17ceb9fe4\": container with ID starting with a359a057666ff7c0814a65b10713f37f11896b3d384aeaac26c679d17ceb9fe4 not found: ID does not exist" containerID="a359a057666ff7c0814a65b10713f37f11896b3d384aeaac26c679d17ceb9fe4"
Sep 08 15:41:41.028128 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.028096    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a359a057666ff7c0814a65b10713f37f11896b3d384aeaac26c679d17ceb9fe4"} err="failed to get container status \"a359a057666ff7c0814a65b10713f37f11896b3d384aeaac26c679d17ceb9fe4\": rpc error: code = NotFound desc = could not find container \"a359a057666ff7c0814a65b10713f37f11896b3d384aeaac26c679d17ceb9fe4\": container with ID starting with a359a057666ff7c0814a65b10713f37f11896b3d384aeaac26c679d17ceb9fe4 not found: ID does not exist"
Sep 08 15:41:41.036817 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.036796    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"]
Sep 08 15:41:41.042395 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.042374    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-4a2c80-openshift-default-584d48f58b-sccd9"]
Sep 08 15:41:41.075343 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.075321    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:41.075343 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.075341    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:41.075442 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.075350    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dldw8\" (UniqueName: \"kubernetes.io/projected/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-kube-api-access-dldw8\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:41.075442 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.075358    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:41.075442 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:41.075367    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:42.432059 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:42.432026    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" path="/var/lib/kubelet/pods/b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6/volumes"
Sep 08 15:41:46.523304 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:41:46.523268    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-serving-cert: secret "maas-api-e2e-ipp-b-9dea65-serving-cert" not found
Sep 08 15:41:46.523902 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:41:46.523363    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-maas-api-tls podName:7df27c42-13ea-46a1-a1e2-ad6a8343ddee nodeName:}" failed. No retries permitted until 2026-09-08 15:41:47.023340008 +0000 UTC m=+1955.109290030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-maas-api-tls") pod "maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs" (UID: "7df27c42-13ea-46a1-a1e2-ad6a8343ddee") : secret "maas-api-e2e-ipp-b-9dea65-serving-cert" not found
Sep 08 15:41:46.523967 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:46.523940    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"]
Sep 08 15:41:46.575013 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:46.574985    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:46.674761 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:46.674728    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:46.891547 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:46.891515    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:46.971902 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:46.971869    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:47.027853 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:41:47.027823    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-serving-cert: secret "maas-api-e2e-ipp-b-9dea65-serving-cert" not found
Sep 08 15:41:47.027976 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:41:47.027898    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-maas-api-tls podName:7df27c42-13ea-46a1-a1e2-ad6a8343ddee nodeName:}" failed. No retries permitted until 2026-09-08 15:41:48.027880838 +0000 UTC m=+1956.113830875 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-maas-api-tls") pod "maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs" (UID: "7df27c42-13ea-46a1-a1e2-ad6a8343ddee") : secret "maas-api-e2e-ipp-b-9dea65-serving-cert" not found
Sep 08 15:41:47.040564 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.040529    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs" podUID="7df27c42-13ea-46a1-a1e2-ad6a8343ddee" containerName="maas-api" containerID="cri-o://762f38c3d2776acb1d6348b32ae528fba51eb79e7aad4cf07a78c4a281f53bb1" gracePeriod=30
Sep 08 15:41:47.252773 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.252744    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:47.289008 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.288987    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:41:47.431895 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.431816    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-maas-api-tls\") pod \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") "
Sep 08 15:41:47.431895 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.431862    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7b6d\" (UniqueName: \"kubernetes.io/projected/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-kube-api-access-p7b6d\") pod \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") "
Sep 08 15:41:47.432119 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.431964    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-metrics-certs\") pod \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") "
Sep 08 15:41:47.432119 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.432003    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-openshift-service-ca\") pod \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\" (UID: \"7df27c42-13ea-46a1-a1e2-ad6a8343ddee\") "
Sep 08 15:41:47.432404 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.432376    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7df27c42-13ea-46a1-a1e2-ad6a8343ddee" (UID: "7df27c42-13ea-46a1-a1e2-ad6a8343ddee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:41:47.433885 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.433859    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7df27c42-13ea-46a1-a1e2-ad6a8343ddee" (UID: "7df27c42-13ea-46a1-a1e2-ad6a8343ddee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:41:47.433976 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.433893    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7df27c42-13ea-46a1-a1e2-ad6a8343ddee" (UID: "7df27c42-13ea-46a1-a1e2-ad6a8343ddee"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:41:47.434017 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.433995    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-kube-api-access-p7b6d" (OuterVolumeSpecName: "kube-api-access-p7b6d") pod "7df27c42-13ea-46a1-a1e2-ad6a8343ddee" (UID: "7df27c42-13ea-46a1-a1e2-ad6a8343ddee"). InnerVolumeSpecName "kube-api-access-p7b6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:41:47.532661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.532637    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-metrics-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:47.532661 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.532661    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-openshift-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:47.533029 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.532670    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-maas-api-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:47.533029 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:47.532678    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7b6d\" (UniqueName: \"kubernetes.io/projected/7df27c42-13ea-46a1-a1e2-ad6a8343ddee-kube-api-access-p7b6d\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:48.045875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:48.045846    2575 generic.go:356] "Generic (PLEG): container finished" podID="7df27c42-13ea-46a1-a1e2-ad6a8343ddee" containerID="762f38c3d2776acb1d6348b32ae528fba51eb79e7aad4cf07a78c4a281f53bb1" exitCode=0
Sep 08 15:41:48.046030 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:48.045891    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"
Sep 08 15:41:48.046030 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:48.045897    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs" event={"ID":"7df27c42-13ea-46a1-a1e2-ad6a8343ddee","Type":"ContainerDied","Data":"762f38c3d2776acb1d6348b32ae528fba51eb79e7aad4cf07a78c4a281f53bb1"}
Sep 08 15:41:48.046030 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:48.045997    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs" event={"ID":"7df27c42-13ea-46a1-a1e2-ad6a8343ddee","Type":"ContainerDied","Data":"f7f01952c713d539d4be279d0020743f66f7b8460f88e3ad792464156ffc093f"}
Sep 08 15:41:48.046030 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:48.046018    2575 scope.go:117] "RemoveContainer" containerID="762f38c3d2776acb1d6348b32ae528fba51eb79e7aad4cf07a78c4a281f53bb1"
Sep 08 15:41:48.055173 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:48.055147    2575 scope.go:117] "RemoveContainer" containerID="762f38c3d2776acb1d6348b32ae528fba51eb79e7aad4cf07a78c4a281f53bb1"
Sep 08 15:41:48.055427 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:41:48.055408    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"762f38c3d2776acb1d6348b32ae528fba51eb79e7aad4cf07a78c4a281f53bb1\": container with ID starting with 762f38c3d2776acb1d6348b32ae528fba51eb79e7aad4cf07a78c4a281f53bb1 not found: ID does not exist" containerID="762f38c3d2776acb1d6348b32ae528fba51eb79e7aad4cf07a78c4a281f53bb1"
Sep 08 15:41:48.055563 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:48.055436    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"762f38c3d2776acb1d6348b32ae528fba51eb79e7aad4cf07a78c4a281f53bb1"} err="failed to get container status \"762f38c3d2776acb1d6348b32ae528fba51eb79e7aad4cf07a78c4a281f53bb1\": rpc error: code = NotFound desc = could not find container \"762f38c3d2776acb1d6348b32ae528fba51eb79e7aad4cf07a78c4a281f53bb1\": container with ID starting with 762f38c3d2776acb1d6348b32ae528fba51eb79e7aad4cf07a78c4a281f53bb1 not found: ID does not exist"
Sep 08 15:41:48.068550 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:48.068519    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"]
Sep 08 15:41:48.075470 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:48.075452    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-9dea65-7fb94c74df-vlfjs"]
Sep 08 15:41:48.432578 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:48.432550    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7df27c42-13ea-46a1-a1e2-ad6a8343ddee" path="/var/lib/kubelet/pods/7df27c42-13ea-46a1-a1e2-ad6a8343ddee/volumes"
Sep 08 15:41:50.547684 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:50.547640    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:41:52.805795 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:52.805751    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:54.141185 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.141142    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:54.243045 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.243003    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:54.735081 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735043    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z"]
Sep 08 15:41:54.735469 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735437    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7df27c42-13ea-46a1-a1e2-ad6a8343ddee" containerName="maas-api"
Sep 08 15:41:54.735536 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735472    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7df27c42-13ea-46a1-a1e2-ad6a8343ddee" containerName="maas-api"
Sep 08 15:41:54.735536 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735527    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a348d026-4378-47a3-a836-97d173458c5b" containerName="main"
Sep 08 15:41:54.735606 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735536    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a348d026-4378-47a3-a836-97d173458c5b" containerName="main"
Sep 08 15:41:54.735606 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735558    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1720dbba-3a6d-4253-819d-193e50c6053b" containerName="maas-api"
Sep 08 15:41:54.735606 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735566    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1720dbba-3a6d-4253-819d-193e50c6053b" containerName="maas-api"
Sep 08 15:41:54.735606 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735575    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy"
Sep 08 15:41:54.735606 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735581    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy"
Sep 08 15:41:54.735606 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735588    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50535ed2-f951-4199-8762-278e26fa046e" containerName="maas-api"
Sep 08 15:41:54.735606 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735594    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="50535ed2-f951-4199-8762-278e26fa046e" containerName="maas-api"
Sep 08 15:41:54.735606 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735600    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerName="istio-proxy"
Sep 08 15:41:54.735606 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735607    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerName="istio-proxy"
Sep 08 15:41:54.735875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735615    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerName="istio-proxy"
Sep 08 15:41:54.735875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735622    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerName="istio-proxy"
Sep 08 15:41:54.735875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735629    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy"
Sep 08 15:41:54.735875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735634    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy"
Sep 08 15:41:54.735875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735695    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy"
Sep 08 15:41:54.735875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735702    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerName="istio-proxy"
Sep 08 15:41:54.735875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735709    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a348d026-4378-47a3-a836-97d173458c5b" containerName="main"
Sep 08 15:41:54.735875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735715    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ff18c8-3caf-4048-b7d8-3c5699caadcc" containerName="istio-proxy"
Sep 08 15:41:54.735875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735720    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bd6d10e-779b-4f2f-94bd-5333cc0093b0" containerName="istio-proxy"
Sep 08 15:41:54.735875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735728    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="50535ed2-f951-4199-8762-278e26fa046e" containerName="maas-api"
Sep 08 15:41:54.735875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735733    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1720dbba-3a6d-4253-819d-193e50c6053b" containerName="maas-api"
Sep 08 15:41:54.735875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.735738    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7df27c42-13ea-46a1-a1e2-ad6a8343ddee" containerName="maas-api"
Sep 08 15:41:54.740052 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.740031    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z"
Sep 08 15:41:54.743948 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.743925    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8qqvg\""
Sep 08 15:41:54.749753 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.749711    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z"]
Sep 08 15:41:54.890998 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.890970    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a427e797-efd3-4f68-acda-858e95d04074-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z\" (UID: \"a427e797-efd3-4f68-acda-858e95d04074\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z"
Sep 08 15:41:54.991747 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.991683    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a427e797-efd3-4f68-acda-858e95d04074-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z\" (UID: \"a427e797-efd3-4f68-acda-858e95d04074\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z"
Sep 08 15:41:54.992272 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:54.992252    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a427e797-efd3-4f68-acda-858e95d04074-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z\" (UID: \"a427e797-efd3-4f68-acda-858e95d04074\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z"
Sep 08 15:41:55.052218 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:55.052194    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z"
Sep 08 15:41:55.331167 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:55.330996    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z"]
Sep 08 15:41:55.334003 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:41:55.333976    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda427e797_efd3_4f68_acda_858e95d04074.slice/crio-223e1118d0c6ac18a6faa9f6d5691bb794de63874454aeec3c738d47e77b0d22 WatchSource:0}: Error finding container 223e1118d0c6ac18a6faa9f6d5691bb794de63874454aeec3c738d47e77b0d22: Status 404 returned error can't find the container with id 223e1118d0c6ac18a6faa9f6d5691bb794de63874454aeec3c738d47e77b0d22
Sep 08 15:41:56.088024 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:56.087993    2575 generic.go:356] "Generic (PLEG): container finished" podID="a427e797-efd3-4f68-acda-858e95d04074" containerID="40f10e5415f807f1c34164890e8c33cfd69f710957a5c3feefeca90dcf64458f" exitCode=0
Sep 08 15:41:56.088143 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:56.088063    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z" event={"ID":"a427e797-efd3-4f68-acda-858e95d04074","Type":"ContainerDied","Data":"40f10e5415f807f1c34164890e8c33cfd69f710957a5c3feefeca90dcf64458f"}
Sep 08 15:41:56.088206 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:56.088138    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z" event={"ID":"a427e797-efd3-4f68-acda-858e95d04074","Type":"ContainerStarted","Data":"223e1118d0c6ac18a6faa9f6d5691bb794de63874454aeec3c738d47e77b0d22"}
Sep 08 15:41:57.223686 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:57.223661    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z"
Sep 08 15:41:57.413298 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:57.413268    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a427e797-efd3-4f68-acda-858e95d04074-maas-api-service-ca\") pod \"a427e797-efd3-4f68-acda-858e95d04074\" (UID: \"a427e797-efd3-4f68-acda-858e95d04074\") "
Sep 08 15:41:57.413654 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:57.413629    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a427e797-efd3-4f68-acda-858e95d04074-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a427e797-efd3-4f68-acda-858e95d04074" (UID: "a427e797-efd3-4f68-acda-858e95d04074"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:41:57.514677 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:57.514649    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a427e797-efd3-4f68-acda-858e95d04074-maas-api-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:41:57.763892 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:57.763783    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:57.864816 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:57.864785    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:41:58.100826 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:58.100765    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z"
Sep 08 15:41:58.100826 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:58.100809    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z" event={"ID":"a427e797-efd3-4f68-acda-858e95d04074","Type":"ContainerDied","Data":"223e1118d0c6ac18a6faa9f6d5691bb794de63874454aeec3c738d47e77b0d22"}
Sep 08 15:41:58.100975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:41:58.100856    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="223e1118d0c6ac18a6faa9f6d5691bb794de63874454aeec3c738d47e77b0d22"
Sep 08 15:42:05.202872 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:05.202780    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:05.306122 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:05.306088    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:05.394381 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:05.394347    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z"]
Sep 08 15:42:05.397743 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:05.397714    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:05.398309 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:05.398291    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2e082e-hgs9z"]
Sep 08 15:42:05.461162 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:05.461102    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:05.794945 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:05.794856    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:06.431413 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:06.431375    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a427e797-efd3-4f68-acda-858e95d04074" path="/var/lib/kubelet/pods/a427e797-efd3-4f68-acda-858e95d04074/volumes"
Sep 08 15:42:08.528421 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:08.528384    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:42:12.062688 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:12.062649    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:12.159647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:12.159612    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:12.316013 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:12.315942    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"]
Sep 08 15:42:12.316181 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:12.316141    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerName="istio-proxy" containerID="cri-o://565fe40be38991611bcf81c4c98a4759a7712834f19801e3d23fdf72e5d2f3d8" gracePeriod=30
Sep 08 15:42:12.332919 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:12.332891    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:12.808781 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:12.808745    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:13.118025 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:13.117997    2575 scope.go:117] "RemoveContainer" containerID="cf3f17f4a86aa8451d8d983e4b701aa186dc75d9ac52c29eccf847e3b799724d"
Sep 08 15:42:17.534454 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.534423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:17.572249 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.572223    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:42:17.598130 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.597994    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:17.691945 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.691912    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdnv5\" (UniqueName: \"kubernetes.io/projected/1761f785-e4f8-4021-a91c-0d4ae9421f88-kube-api-access-hdnv5\") pod \"1761f785-e4f8-4021-a91c-0d4ae9421f88\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") "
Sep 08 15:42:17.692116 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.691952    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-data\") pod \"1761f785-e4f8-4021-a91c-0d4ae9421f88\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") "
Sep 08 15:42:17.692116 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.691994    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-credential-socket\") pod \"1761f785-e4f8-4021-a91c-0d4ae9421f88\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") "
Sep 08 15:42:17.692116 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692038    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-envoy\") pod \"1761f785-e4f8-4021-a91c-0d4ae9421f88\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") "
Sep 08 15:42:17.692116 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692057    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-workload-certs\") pod \"1761f785-e4f8-4021-a91c-0d4ae9421f88\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") "
Sep 08 15:42:17.692116 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692080    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-token\") pod \"1761f785-e4f8-4021-a91c-0d4ae9421f88\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") "
Sep 08 15:42:17.692391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692118    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-workload-socket\") pod \"1761f785-e4f8-4021-a91c-0d4ae9421f88\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") "
Sep 08 15:42:17.692391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692162    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-podinfo\") pod \"1761f785-e4f8-4021-a91c-0d4ae9421f88\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") "
Sep 08 15:42:17.692391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692203    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1761f785-e4f8-4021-a91c-0d4ae9421f88-istiod-ca-cert\") pod \"1761f785-e4f8-4021-a91c-0d4ae9421f88\" (UID: \"1761f785-e4f8-4021-a91c-0d4ae9421f88\") "
Sep 08 15:42:17.692391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692228    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-data" (OuterVolumeSpecName: "istio-data") pod "1761f785-e4f8-4021-a91c-0d4ae9421f88" (UID: "1761f785-e4f8-4021-a91c-0d4ae9421f88"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:42:17.692391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692333    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1761f785-e4f8-4021-a91c-0d4ae9421f88" (UID: "1761f785-e4f8-4021-a91c-0d4ae9421f88"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:42:17.692391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692347    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1761f785-e4f8-4021-a91c-0d4ae9421f88" (UID: "1761f785-e4f8-4021-a91c-0d4ae9421f88"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:42:17.692391 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692390    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1761f785-e4f8-4021-a91c-0d4ae9421f88" (UID: "1761f785-e4f8-4021-a91c-0d4ae9421f88"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:42:17.692777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692652    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-workload-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:42:17.692777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692671    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-data\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:42:17.692777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692667    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1761f785-e4f8-4021-a91c-0d4ae9421f88-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1761f785-e4f8-4021-a91c-0d4ae9421f88" (UID: "1761f785-e4f8-4021-a91c-0d4ae9421f88"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:42:17.692777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692685    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-credential-socket\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:42:17.692777 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.692699    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-workload-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:42:17.694421 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.694390    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1761f785-e4f8-4021-a91c-0d4ae9421f88" (UID: "1761f785-e4f8-4021-a91c-0d4ae9421f88"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:42:17.694647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.694426    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1761f785-e4f8-4021-a91c-0d4ae9421f88" (UID: "1761f785-e4f8-4021-a91c-0d4ae9421f88"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:42:17.694647 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.694437    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1761f785-e4f8-4021-a91c-0d4ae9421f88-kube-api-access-hdnv5" (OuterVolumeSpecName: "kube-api-access-hdnv5") pod "1761f785-e4f8-4021-a91c-0d4ae9421f88" (UID: "1761f785-e4f8-4021-a91c-0d4ae9421f88"). InnerVolumeSpecName "kube-api-access-hdnv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:42:17.694840 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.694820    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-token" (OuterVolumeSpecName: "istio-token") pod "1761f785-e4f8-4021-a91c-0d4ae9421f88" (UID: "1761f785-e4f8-4021-a91c-0d4ae9421f88"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:42:17.793535 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.793479    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-podinfo\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:42:17.793535 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.793533    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1761f785-e4f8-4021-a91c-0d4ae9421f88-istiod-ca-cert\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:42:17.793535 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.793543    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdnv5\" (UniqueName: \"kubernetes.io/projected/1761f785-e4f8-4021-a91c-0d4ae9421f88-kube-api-access-hdnv5\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:42:17.793749 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.793554    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-envoy\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:42:17.793749 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:17.793562    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1761f785-e4f8-4021-a91c-0d4ae9421f88-istio-token\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:42:18.201803 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:18.201768    2575 generic.go:356] "Generic (PLEG): container finished" podID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerID="565fe40be38991611bcf81c4c98a4759a7712834f19801e3d23fdf72e5d2f3d8" exitCode=0
Sep 08 15:42:18.201933 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:18.201812    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"
Sep 08 15:42:18.201933 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:18.201847    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" event={"ID":"1761f785-e4f8-4021-a91c-0d4ae9421f88","Type":"ContainerDied","Data":"565fe40be38991611bcf81c4c98a4759a7712834f19801e3d23fdf72e5d2f3d8"}
Sep 08 15:42:18.201933 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:18.201898    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf" event={"ID":"1761f785-e4f8-4021-a91c-0d4ae9421f88","Type":"ContainerDied","Data":"23375f357a7f35cb6b2a7360e5a101abae5dc70c3c47f5395b1ee0451be2ffff"}
Sep 08 15:42:18.201933 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:18.201923    2575 scope.go:117] "RemoveContainer" containerID="565fe40be38991611bcf81c4c98a4759a7712834f19801e3d23fdf72e5d2f3d8"
Sep 08 15:42:18.210720 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:18.210704    2575 scope.go:117] "RemoveContainer" containerID="89ae883fa70679ded8578ee0154fe1d0773a187d027b79060ee6580efa1b62b1"
Sep 08 15:42:18.218828 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:18.218808    2575 scope.go:117] "RemoveContainer" containerID="565fe40be38991611bcf81c4c98a4759a7712834f19801e3d23fdf72e5d2f3d8"
Sep 08 15:42:18.219054 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:42:18.219039    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"565fe40be38991611bcf81c4c98a4759a7712834f19801e3d23fdf72e5d2f3d8\": container with ID starting with 565fe40be38991611bcf81c4c98a4759a7712834f19801e3d23fdf72e5d2f3d8 not found: ID does not exist" containerID="565fe40be38991611bcf81c4c98a4759a7712834f19801e3d23fdf72e5d2f3d8"
Sep 08 15:42:18.219108 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:18.219062    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"565fe40be38991611bcf81c4c98a4759a7712834f19801e3d23fdf72e5d2f3d8"} err="failed to get container status \"565fe40be38991611bcf81c4c98a4759a7712834f19801e3d23fdf72e5d2f3d8\": rpc error: code = NotFound desc = could not find container \"565fe40be38991611bcf81c4c98a4759a7712834f19801e3d23fdf72e5d2f3d8\": container with ID starting with 565fe40be38991611bcf81c4c98a4759a7712834f19801e3d23fdf72e5d2f3d8 not found: ID does not exist"
Sep 08 15:42:18.219108 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:18.219078    2575 scope.go:117] "RemoveContainer" containerID="89ae883fa70679ded8578ee0154fe1d0773a187d027b79060ee6580efa1b62b1"
Sep 08 15:42:18.219331 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:42:18.219312    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89ae883fa70679ded8578ee0154fe1d0773a187d027b79060ee6580efa1b62b1\": container with ID starting with 89ae883fa70679ded8578ee0154fe1d0773a187d027b79060ee6580efa1b62b1 not found: ID does not exist" containerID="89ae883fa70679ded8578ee0154fe1d0773a187d027b79060ee6580efa1b62b1"
Sep 08 15:42:18.219376 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:18.219336    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89ae883fa70679ded8578ee0154fe1d0773a187d027b79060ee6580efa1b62b1"} err="failed to get container status \"89ae883fa70679ded8578ee0154fe1d0773a187d027b79060ee6580efa1b62b1\": rpc error: code = NotFound desc = could not find container \"89ae883fa70679ded8578ee0154fe1d0773a187d027b79060ee6580efa1b62b1\": container with ID starting with 89ae883fa70679ded8578ee0154fe1d0773a187d027b79060ee6580efa1b62b1 not found: ID does not exist"
Sep 08 15:42:18.226091 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:18.226071    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"]
Sep 08 15:42:18.234854 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:18.234831    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2e082e-openshift-default-79f8ff74f6-d96gf"]
Sep 08 15:42:18.431567 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:18.431545    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" path="/var/lib/kubelet/pods/1761f785-e4f8-4021-a91c-0d4ae9421f88/volumes"
Sep 08 15:42:23.355930 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.355897    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"]
Sep 08 15:42:23.356298 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.356092    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg" podUID="4a92a8fe-14f5-4bf0-867b-fe6114321b07" containerName="maas-api" containerID="cri-o://29def99d830c0f7a02be834dad3d31bd54d8169461ed53976de5b12af1242494" gracePeriod=30
Sep 08 15:42:23.395385 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.395295    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:23.500932 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.500895    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:23.599325 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.599298    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:42:23.601616 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.601593    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:23.712700 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.712617    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:23.742762 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.742736    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-metrics-certs\") pod \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") "
Sep 08 15:42:23.742894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.742846    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-maas-api-tls\") pod \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") "
Sep 08 15:42:23.742894 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.742873    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92a8fe-14f5-4bf0-867b-fe6114321b07-openshift-service-ca\") pod \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") "
Sep 08 15:42:23.743012 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.742904    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dqvv\" (UniqueName: \"kubernetes.io/projected/4a92a8fe-14f5-4bf0-867b-fe6114321b07-kube-api-access-6dqvv\") pod \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\" (UID: \"4a92a8fe-14f5-4bf0-867b-fe6114321b07\") "
Sep 08 15:42:23.743361 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.743334    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a92a8fe-14f5-4bf0-867b-fe6114321b07-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4a92a8fe-14f5-4bf0-867b-fe6114321b07" (UID: "4a92a8fe-14f5-4bf0-867b-fe6114321b07"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:42:23.745314 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.745282    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4a92a8fe-14f5-4bf0-867b-fe6114321b07" (UID: "4a92a8fe-14f5-4bf0-867b-fe6114321b07"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:42:23.745439 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.745356    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a92a8fe-14f5-4bf0-867b-fe6114321b07-kube-api-access-6dqvv" (OuterVolumeSpecName: "kube-api-access-6dqvv") pod "4a92a8fe-14f5-4bf0-867b-fe6114321b07" (UID: "4a92a8fe-14f5-4bf0-867b-fe6114321b07"). InnerVolumeSpecName "kube-api-access-6dqvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:42:23.745605 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.745583    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4a92a8fe-14f5-4bf0-867b-fe6114321b07" (UID: "4a92a8fe-14f5-4bf0-867b-fe6114321b07"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:42:23.844538 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.844486    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-maas-api-tls\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:42:23.844627 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.844540    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a92a8fe-14f5-4bf0-867b-fe6114321b07-openshift-service-ca\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:42:23.844627 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.844555    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dqvv\" (UniqueName: \"kubernetes.io/projected/4a92a8fe-14f5-4bf0-867b-fe6114321b07-kube-api-access-6dqvv\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:42:23.844627 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:23.844568    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a92a8fe-14f5-4bf0-867b-fe6114321b07-metrics-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:42:24.233327 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:24.233295    2575 generic.go:356] "Generic (PLEG): container finished" podID="4a92a8fe-14f5-4bf0-867b-fe6114321b07" containerID="29def99d830c0f7a02be834dad3d31bd54d8169461ed53976de5b12af1242494" exitCode=0
Sep 08 15:42:24.233541 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:24.233340    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"
Sep 08 15:42:24.233541 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:24.233370    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg" event={"ID":"4a92a8fe-14f5-4bf0-867b-fe6114321b07","Type":"ContainerDied","Data":"29def99d830c0f7a02be834dad3d31bd54d8169461ed53976de5b12af1242494"}
Sep 08 15:42:24.233541 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:24.233414    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg" event={"ID":"4a92a8fe-14f5-4bf0-867b-fe6114321b07","Type":"ContainerDied","Data":"841f3f01f9239e15b5d94f13217706fb73dfb605d09d4f2452b1a11edf73c955"}
Sep 08 15:42:24.233541 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:24.233449    2575 scope.go:117] "RemoveContainer" containerID="29def99d830c0f7a02be834dad3d31bd54d8169461ed53976de5b12af1242494"
Sep 08 15:42:24.243353 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:24.243333    2575 scope.go:117] "RemoveContainer" containerID="29def99d830c0f7a02be834dad3d31bd54d8169461ed53976de5b12af1242494"
Sep 08 15:42:24.243699 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:42:24.243677    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29def99d830c0f7a02be834dad3d31bd54d8169461ed53976de5b12af1242494\": container with ID starting with 29def99d830c0f7a02be834dad3d31bd54d8169461ed53976de5b12af1242494 not found: ID does not exist" containerID="29def99d830c0f7a02be834dad3d31bd54d8169461ed53976de5b12af1242494"
Sep 08 15:42:24.243772 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:24.243707    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29def99d830c0f7a02be834dad3d31bd54d8169461ed53976de5b12af1242494"} err="failed to get container status \"29def99d830c0f7a02be834dad3d31bd54d8169461ed53976de5b12af1242494\": rpc error: code = NotFound desc = could not find container \"29def99d830c0f7a02be834dad3d31bd54d8169461ed53976de5b12af1242494\": container with ID starting with 29def99d830c0f7a02be834dad3d31bd54d8169461ed53976de5b12af1242494 not found: ID does not exist"
Sep 08 15:42:24.255932 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:24.255904    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"]
Sep 08 15:42:24.263255 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:24.263231    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bc9cdf-5bbb99db79-p5zpg"]
Sep 08 15:42:24.432454 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:24.432417    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a92a8fe-14f5-4bf0-867b-fe6114321b07" path="/var/lib/kubelet/pods/4a92a8fe-14f5-4bf0-867b-fe6114321b07/volumes"
Sep 08 15:42:26.133758 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:26.133721    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:42:27.327604 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:27.327565    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:31.060977 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:31.060940    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:31.161925 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:31.161893    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:31.339172 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:31.339070    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:42:36.147080 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:42:36.147040    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:43:56.342101 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:43:56.342015    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:57.134995 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:43:57.134965    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:43:57.377250 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:43:57.377219    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:43:58.921622 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:43:58.921590    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:01.847808 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:44:01.847716    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:03.405586 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:44:03.405551    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:03.479221 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:44:03.479189    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:12.491052 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:44:12.491024    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:44:12.500583 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:44:12.500562    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:44:20.940257 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:44:20.940224    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:44:28.932672 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:44:28.932630    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:44:36.940978 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:44:36.940947    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:44:51.612861 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:44:51.612814    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:44:52.934358 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:44:52.934320    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:44:58.875019 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:44:58.874990    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:45:00.301768 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:45:00.301730    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:46:29.153549 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:29.153513    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:46:35.239713 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:35.239642    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:46:50.930930 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.930892    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"]
Sep 08 15:46:50.931368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.931325    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerName="istio-proxy"
Sep 08 15:46:50.931368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.931342    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerName="istio-proxy"
Sep 08 15:46:50.931368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.931354    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerName="istio-proxy"
Sep 08 15:46:50.931368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.931363    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerName="istio-proxy"
Sep 08 15:46:50.931530 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.931387    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a92a8fe-14f5-4bf0-867b-fe6114321b07" containerName="maas-api"
Sep 08 15:46:50.931530 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.931396    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a92a8fe-14f5-4bf0-867b-fe6114321b07" containerName="maas-api"
Sep 08 15:46:50.931530 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.931413    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a427e797-efd3-4f68-acda-858e95d04074" containerName="revoke-keys"
Sep 08 15:46:50.931530 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.931421    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a427e797-efd3-4f68-acda-858e95d04074" containerName="revoke-keys"
Sep 08 15:46:50.931530 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.931511    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerName="istio-proxy"
Sep 08 15:46:50.931530 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.931521    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a92a8fe-14f5-4bf0-867b-fe6114321b07" containerName="maas-api"
Sep 08 15:46:50.931530 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.931531    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a427e797-efd3-4f68-acda-858e95d04074" containerName="revoke-keys"
Sep 08 15:46:50.931827 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.931538    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b34bfc4d-2e1d-4a5d-a2d5-68e05c5b31d6" containerName="istio-proxy"
Sep 08 15:46:50.934614 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.934596    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:50.937192 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.937163    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-dd17d142-kserve-self-signed-certs\""
Sep 08 15:46:50.945340 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:50.945317    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"]
Sep 08 15:46:51.000707 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.000677    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:46:51.070741 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.070707    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52578d94-6740-4fd2-b11e-eeddb3c37d53-tls-certs\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.070903 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.070765    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-model-cache\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.070903 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.070866    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-dshm\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.070981 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.070900    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-tmp-dir\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.070981 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.070943    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-home\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.070981 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.070976    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rzxb\" (UniqueName: \"kubernetes.io/projected/52578d94-6740-4fd2-b11e-eeddb3c37d53-kube-api-access-7rzxb\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.171905 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.171872    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52578d94-6740-4fd2-b11e-eeddb3c37d53-tls-certs\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.172068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.171928    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-model-cache\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.172068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.171971    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-dshm\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.172068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.171988    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-tmp-dir\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.172068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.172019    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-home\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.172068 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.172063    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rzxb\" (UniqueName: \"kubernetes.io/projected/52578d94-6740-4fd2-b11e-eeddb3c37d53-kube-api-access-7rzxb\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.172417 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.172396    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-tmp-dir\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.172539 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.172441    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-model-cache\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.172539 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.172509    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-home\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.174198 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.174176    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-dshm\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.174511 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.174478    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52578d94-6740-4fd2-b11e-eeddb3c37d53-tls-certs\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.181775 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.181724    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rzxb\" (UniqueName: \"kubernetes.io/projected/52578d94-6740-4fd2-b11e-eeddb3c37d53-kube-api-access-7rzxb\") pod \"e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") " pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.243942 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.243916    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:46:51.518368 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:51.518345    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"]
Sep 08 15:46:51.520535 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:46:51.520509    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52578d94_6740_4fd2_b11e_eeddb3c37d53.slice/crio-5e4db0adcb3402ed28cdf61dfd290e95e5c5f7b89db127fb24926843d063105b WatchSource:0}: Error finding container 5e4db0adcb3402ed28cdf61dfd290e95e5c5f7b89db127fb24926843d063105b: Status 404 returned error can't find the container with id 5e4db0adcb3402ed28cdf61dfd290e95e5c5f7b89db127fb24926843d063105b
Sep 08 15:46:52.514649 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:52.514614    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf" event={"ID":"52578d94-6740-4fd2-b11e-eeddb3c37d53","Type":"ContainerStarted","Data":"151bb0b8b5844761f7f9d203011cbbf390ac6324b589df8618d03a7ec2352270"}
Sep 08 15:46:52.514649 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:52.514654    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf" event={"ID":"52578d94-6740-4fd2-b11e-eeddb3c37d53","Type":"ContainerStarted","Data":"5e4db0adcb3402ed28cdf61dfd290e95e5c5f7b89db127fb24926843d063105b"}
Sep 08 15:46:52.533560 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:52.533512    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf" podStartSLOduration=2.533485487 podStartE2EDuration="2.533485487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:46:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:46:52.53241074 +0000 UTC m=+2260.618360781" watchObservedRunningTime="2026-09-08 15:46:52.533485487 +0000 UTC m=+2260.619435529"
Sep 08 15:46:53.749122 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:53.749085    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:46:56.741957 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.741919    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"]
Sep 08 15:46:56.742450 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.742433    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1761f785-e4f8-4021-a91c-0d4ae9421f88" containerName="istio-proxy"
Sep 08 15:46:56.745621 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.745605    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.749288 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.749270    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-dd17d142-kserve-self-signed-certs\""
Sep 08 15:46:56.761118 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.761096    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"]
Sep 08 15:46:56.780992 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.780966    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:46:56.824596 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.824574    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43b1e82a-492b-442f-9356-3f103e456ba1-tls-certs\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.824691 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.824630    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-tmp-dir\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.824741 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.824684    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-home\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.824741 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.824724    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-model-cache\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.824818 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.824763    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h62p\" (UniqueName: \"kubernetes.io/projected/43b1e82a-492b-442f-9356-3f103e456ba1-kube-api-access-7h62p\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.824818 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.824781    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-dshm\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.925535 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.925486    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h62p\" (UniqueName: \"kubernetes.io/projected/43b1e82a-492b-442f-9356-3f103e456ba1-kube-api-access-7h62p\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.925663 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.925540    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-dshm\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.925663 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.925572    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43b1e82a-492b-442f-9356-3f103e456ba1-tls-certs\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.925663 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.925622    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-tmp-dir\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.925843 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.925781    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-home\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.925843 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.925829    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-model-cache\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.926048 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.926027    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-tmp-dir\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.926115 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.926082    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-home\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.926207 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.926187    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-model-cache\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.927779 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.927753    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-dshm\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.928096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.928077    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43b1e82a-492b-442f-9356-3f103e456ba1-tls-certs\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:56.934140 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:56.934119    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h62p\" (UniqueName: \"kubernetes.io/projected/43b1e82a-492b-442f-9356-3f103e456ba1-kube-api-access-7h62p\") pod \"e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") " pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:57.055880 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:57.055819    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:46:57.329951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:57.329919    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"]
Sep 08 15:46:57.331443 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:46:57.331417    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43b1e82a_492b_442f_9356_3f103e456ba1.slice/crio-9175049bc993ee69de4d076e4297b32e337660dc78af106026123996b5a49c94 WatchSource:0}: Error finding container 9175049bc993ee69de4d076e4297b32e337660dc78af106026123996b5a49c94: Status 404 returned error can't find the container with id 9175049bc993ee69de4d076e4297b32e337660dc78af106026123996b5a49c94
Sep 08 15:46:57.539272 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:57.539238    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w" event={"ID":"43b1e82a-492b-442f-9356-3f103e456ba1","Type":"ContainerStarted","Data":"c8460399d4865fea3aae75936e859e35f98d87f191af4d592b4340d56a60ab1b"}
Sep 08 15:46:57.539429 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:57.539279    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w" event={"ID":"43b1e82a-492b-442f-9356-3f103e456ba1","Type":"ContainerStarted","Data":"9175049bc993ee69de4d076e4297b32e337660dc78af106026123996b5a49c94"}
Sep 08 15:46:57.561085 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:46:57.561041    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w" podStartSLOduration=1.561026804 podStartE2EDuration="1.561026804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:46:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:46:57.560285898 +0000 UTC m=+2265.646235951" watchObservedRunningTime="2026-09-08 15:46:57.561026804 +0000 UTC m=+2265.646976894"
Sep 08 15:47:01.244330 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:01.244284    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:47:01.244763 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:01.244344    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:47:01.256728 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:01.256705    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:47:01.569293 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:01.569263    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:47:05.430096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:05.430056    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:47:07.056049 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:07.056014    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:47:07.056429 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:07.056257    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:47:07.068343 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:07.068320    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:47:07.597435 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:07.597406    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:47:32.445266 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:32.445232    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"]
Sep 08 15:47:32.486966 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:32.486929    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:34.260858 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:34.260821    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"]
Sep 08 15:47:34.284160 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:34.284130    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:47:34.440846 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:34.440817    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:47:43.035791 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:43.035742    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:47:47.465636 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.465578    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf" podUID="52578d94-6740-4fd2-b11e-eeddb3c37d53" containerName="main" containerID="cri-o://151bb0b8b5844761f7f9d203011cbbf390ac6324b589df8618d03a7ec2352270" gracePeriod=45
Sep 08 15:47:47.722034 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.721973    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:47:47.785525 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.785456    2575 generic.go:356] "Generic (PLEG): container finished" podID="52578d94-6740-4fd2-b11e-eeddb3c37d53" containerID="151bb0b8b5844761f7f9d203011cbbf390ac6324b589df8618d03a7ec2352270" exitCode=0
Sep 08 15:47:47.785690 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.785544    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf" event={"ID":"52578d94-6740-4fd2-b11e-eeddb3c37d53","Type":"ContainerDied","Data":"151bb0b8b5844761f7f9d203011cbbf390ac6324b589df8618d03a7ec2352270"}
Sep 08 15:47:47.785690 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.785587    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"
Sep 08 15:47:47.785690 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.785623    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf" event={"ID":"52578d94-6740-4fd2-b11e-eeddb3c37d53","Type":"ContainerDied","Data":"5e4db0adcb3402ed28cdf61dfd290e95e5c5f7b89db127fb24926843d063105b"}
Sep 08 15:47:47.785690 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.785657    2575 scope.go:117] "RemoveContainer" containerID="151bb0b8b5844761f7f9d203011cbbf390ac6324b589df8618d03a7ec2352270"
Sep 08 15:47:47.796847 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.796825    2575 scope.go:117] "RemoveContainer" containerID="151bb0b8b5844761f7f9d203011cbbf390ac6324b589df8618d03a7ec2352270"
Sep 08 15:47:47.797172 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:47:47.797147    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"151bb0b8b5844761f7f9d203011cbbf390ac6324b589df8618d03a7ec2352270\": container with ID starting with 151bb0b8b5844761f7f9d203011cbbf390ac6324b589df8618d03a7ec2352270 not found: ID does not exist" containerID="151bb0b8b5844761f7f9d203011cbbf390ac6324b589df8618d03a7ec2352270"
Sep 08 15:47:47.797318 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.797177    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"151bb0b8b5844761f7f9d203011cbbf390ac6324b589df8618d03a7ec2352270"} err="failed to get container status \"151bb0b8b5844761f7f9d203011cbbf390ac6324b589df8618d03a7ec2352270\": rpc error: code = NotFound desc = could not find container \"151bb0b8b5844761f7f9d203011cbbf390ac6324b589df8618d03a7ec2352270\": container with ID starting with 151bb0b8b5844761f7f9d203011cbbf390ac6324b589df8618d03a7ec2352270 not found: ID does not exist"
Sep 08 15:47:47.904223 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.904199    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-home\") pod \"52578d94-6740-4fd2-b11e-eeddb3c37d53\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") "
Sep 08 15:47:47.904372 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.904234    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rzxb\" (UniqueName: \"kubernetes.io/projected/52578d94-6740-4fd2-b11e-eeddb3c37d53-kube-api-access-7rzxb\") pod \"52578d94-6740-4fd2-b11e-eeddb3c37d53\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") "
Sep 08 15:47:47.904372 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.904342    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52578d94-6740-4fd2-b11e-eeddb3c37d53-tls-certs\") pod \"52578d94-6740-4fd2-b11e-eeddb3c37d53\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") "
Sep 08 15:47:47.904372 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.904367    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-tmp-dir\") pod \"52578d94-6740-4fd2-b11e-eeddb3c37d53\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") "
Sep 08 15:47:47.904591 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.904400    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-dshm\") pod \"52578d94-6740-4fd2-b11e-eeddb3c37d53\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") "
Sep 08 15:47:47.904591 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.904440    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-model-cache\") pod \"52578d94-6740-4fd2-b11e-eeddb3c37d53\" (UID: \"52578d94-6740-4fd2-b11e-eeddb3c37d53\") "
Sep 08 15:47:47.904591 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.904521    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-home" (OuterVolumeSpecName: "home") pod "52578d94-6740-4fd2-b11e-eeddb3c37d53" (UID: "52578d94-6740-4fd2-b11e-eeddb3c37d53"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:47:47.904791 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.904768    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "52578d94-6740-4fd2-b11e-eeddb3c37d53" (UID: "52578d94-6740-4fd2-b11e-eeddb3c37d53"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:47:47.904791 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.904784    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:47:47.904993 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.904971    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-model-cache" (OuterVolumeSpecName: "model-cache") pod "52578d94-6740-4fd2-b11e-eeddb3c37d53" (UID: "52578d94-6740-4fd2-b11e-eeddb3c37d53"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:47:47.906883 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.906850    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52578d94-6740-4fd2-b11e-eeddb3c37d53-kube-api-access-7rzxb" (OuterVolumeSpecName: "kube-api-access-7rzxb") pod "52578d94-6740-4fd2-b11e-eeddb3c37d53" (UID: "52578d94-6740-4fd2-b11e-eeddb3c37d53"). InnerVolumeSpecName "kube-api-access-7rzxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:47:47.906999 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.906971    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52578d94-6740-4fd2-b11e-eeddb3c37d53-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "52578d94-6740-4fd2-b11e-eeddb3c37d53" (UID: "52578d94-6740-4fd2-b11e-eeddb3c37d53"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:47:47.907224 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:47.907200    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-dshm" (OuterVolumeSpecName: "dshm") pod "52578d94-6740-4fd2-b11e-eeddb3c37d53" (UID: "52578d94-6740-4fd2-b11e-eeddb3c37d53"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:47:48.005875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:48.005811    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:47:48.005875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:48.005837    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:47:48.005875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:48.005848    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rzxb\" (UniqueName: \"kubernetes.io/projected/52578d94-6740-4fd2-b11e-eeddb3c37d53-kube-api-access-7rzxb\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:47:48.005875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:48.005856    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52578d94-6740-4fd2-b11e-eeddb3c37d53-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:47:48.005875 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:48.005865    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52578d94-6740-4fd2-b11e-eeddb3c37d53-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:47:48.106500 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:48.106459    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"]
Sep 08 15:47:48.110283 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:48.110258    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-dd17d142-kserve-7d54b9b58c-c9qvf"]
Sep 08 15:47:48.432734 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:48.432700    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52578d94-6740-4fd2-b11e-eeddb3c37d53" path="/var/lib/kubelet/pods/52578d94-6740-4fd2-b11e-eeddb3c37d53/volumes"
Sep 08 15:47:49.280928 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.280886    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w" podUID="43b1e82a-492b-442f-9356-3f103e456ba1" containerName="main" containerID="cri-o://c8460399d4865fea3aae75936e859e35f98d87f191af4d592b4340d56a60ab1b" gracePeriod=45
Sep 08 15:47:49.522086 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.522063    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:47:49.619600 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.619573    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-home\") pod \"43b1e82a-492b-442f-9356-3f103e456ba1\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") "
Sep 08 15:47:49.619747 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.619622    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43b1e82a-492b-442f-9356-3f103e456ba1-tls-certs\") pod \"43b1e82a-492b-442f-9356-3f103e456ba1\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") "
Sep 08 15:47:49.619747 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.619646    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-tmp-dir\") pod \"43b1e82a-492b-442f-9356-3f103e456ba1\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") "
Sep 08 15:47:49.619747 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.619741    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-model-cache\") pod \"43b1e82a-492b-442f-9356-3f103e456ba1\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") "
Sep 08 15:47:49.619899 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.619781    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h62p\" (UniqueName: \"kubernetes.io/projected/43b1e82a-492b-442f-9356-3f103e456ba1-kube-api-access-7h62p\") pod \"43b1e82a-492b-442f-9356-3f103e456ba1\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") "
Sep 08 15:47:49.619899 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.619817    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-dshm\") pod \"43b1e82a-492b-442f-9356-3f103e456ba1\" (UID: \"43b1e82a-492b-442f-9356-3f103e456ba1\") "
Sep 08 15:47:49.620013 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.619893    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-home" (OuterVolumeSpecName: "home") pod "43b1e82a-492b-442f-9356-3f103e456ba1" (UID: "43b1e82a-492b-442f-9356-3f103e456ba1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:47:49.620013 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.619922    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "43b1e82a-492b-442f-9356-3f103e456ba1" (UID: "43b1e82a-492b-442f-9356-3f103e456ba1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:47:49.620116 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.620009    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-model-cache" (OuterVolumeSpecName: "model-cache") pod "43b1e82a-492b-442f-9356-3f103e456ba1" (UID: "43b1e82a-492b-442f-9356-3f103e456ba1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:47:49.620210 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.620192    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-model-cache\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:47:49.620268 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.620215    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-home\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:47:49.620268 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.620229    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-tmp-dir\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:47:49.621788 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.621761    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-dshm" (OuterVolumeSpecName: "dshm") pod "43b1e82a-492b-442f-9356-3f103e456ba1" (UID: "43b1e82a-492b-442f-9356-3f103e456ba1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:47:49.621890 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.621824    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43b1e82a-492b-442f-9356-3f103e456ba1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "43b1e82a-492b-442f-9356-3f103e456ba1" (UID: "43b1e82a-492b-442f-9356-3f103e456ba1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:47:49.621890 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.621846    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43b1e82a-492b-442f-9356-3f103e456ba1-kube-api-access-7h62p" (OuterVolumeSpecName: "kube-api-access-7h62p") pod "43b1e82a-492b-442f-9356-3f103e456ba1" (UID: "43b1e82a-492b-442f-9356-3f103e456ba1"). InnerVolumeSpecName "kube-api-access-7h62p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:47:49.721595 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.721566    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43b1e82a-492b-442f-9356-3f103e456ba1-tls-certs\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:47:49.721595 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.721589    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h62p\" (UniqueName: \"kubernetes.io/projected/43b1e82a-492b-442f-9356-3f103e456ba1-kube-api-access-7h62p\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:47:49.721595 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.721598    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43b1e82a-492b-442f-9356-3f103e456ba1-dshm\") on node \"ip-10-0-132-76.ec2.internal\" DevicePath \"\""
Sep 08 15:47:49.799724 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.799690    2575 generic.go:356] "Generic (PLEG): container finished" podID="43b1e82a-492b-442f-9356-3f103e456ba1" containerID="c8460399d4865fea3aae75936e859e35f98d87f191af4d592b4340d56a60ab1b" exitCode=0
Sep 08 15:47:49.799881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.799737    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"
Sep 08 15:47:49.799881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.799749    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w" event={"ID":"43b1e82a-492b-442f-9356-3f103e456ba1","Type":"ContainerDied","Data":"c8460399d4865fea3aae75936e859e35f98d87f191af4d592b4340d56a60ab1b"}
Sep 08 15:47:49.799881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.799788    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w" event={"ID":"43b1e82a-492b-442f-9356-3f103e456ba1","Type":"ContainerDied","Data":"9175049bc993ee69de4d076e4297b32e337660dc78af106026123996b5a49c94"}
Sep 08 15:47:49.799881 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.799810    2575 scope.go:117] "RemoveContainer" containerID="c8460399d4865fea3aae75936e859e35f98d87f191af4d592b4340d56a60ab1b"
Sep 08 15:47:49.808987 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.808968    2575 scope.go:117] "RemoveContainer" containerID="c8460399d4865fea3aae75936e859e35f98d87f191af4d592b4340d56a60ab1b"
Sep 08 15:47:49.809247 ip-10-0-132-76 kubenswrapper[2575]: E0908 15:47:49.809227    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8460399d4865fea3aae75936e859e35f98d87f191af4d592b4340d56a60ab1b\": container with ID starting with c8460399d4865fea3aae75936e859e35f98d87f191af4d592b4340d56a60ab1b not found: ID does not exist" containerID="c8460399d4865fea3aae75936e859e35f98d87f191af4d592b4340d56a60ab1b"
Sep 08 15:47:49.809312 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.809254    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8460399d4865fea3aae75936e859e35f98d87f191af4d592b4340d56a60ab1b"} err="failed to get container status \"c8460399d4865fea3aae75936e859e35f98d87f191af4d592b4340d56a60ab1b\": rpc error: code = NotFound desc = could not find container \"c8460399d4865fea3aae75936e859e35f98d87f191af4d592b4340d56a60ab1b\": container with ID starting with c8460399d4865fea3aae75936e859e35f98d87f191af4d592b4340d56a60ab1b not found: ID does not exist"
Sep 08 15:47:49.823565 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.823539    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"]
Sep 08 15:47:49.827271 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.827250    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-dd17d142-kserve-765d65f967-9d26w"]
Sep 08 15:47:49.838683 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:49.838655    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:47:50.431555 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:50.431521    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43b1e82a-492b-442f-9356-3f103e456ba1" path="/var/lib/kubelet/pods/43b1e82a-492b-442f-9356-3f103e456ba1/volumes"
Sep 08 15:47:54.581432 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.581399    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"]
Sep 08 15:47:54.581914 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.581889    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52578d94-6740-4fd2-b11e-eeddb3c37d53" containerName="main"
Sep 08 15:47:54.581984 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.581917    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="52578d94-6740-4fd2-b11e-eeddb3c37d53" containerName="main"
Sep 08 15:47:54.581984 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.581942    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43b1e82a-492b-442f-9356-3f103e456ba1" containerName="main"
Sep 08 15:47:54.581984 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.581950    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="43b1e82a-492b-442f-9356-3f103e456ba1" containerName="main"
Sep 08 15:47:54.582096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.582025    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="52578d94-6740-4fd2-b11e-eeddb3c37d53" containerName="main"
Sep 08 15:47:54.582096 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.582034    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="43b1e82a-492b-442f-9356-3f103e456ba1" containerName="main"
Sep 08 15:47:54.586246 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.586229    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.588877 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.588853    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:47:54.589030 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.588884    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 08 15:47:54.590232 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.590211    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 08 15:47:54.590332 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.590258    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hrbm8\""
Sep 08 15:47:54.590332 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.590294    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 08 15:47:54.594700 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.594666    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"]
Sep 08 15:47:54.666962 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.666921    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-784dj\" (UniqueName: \"kubernetes.io/projected/708bd367-46ef-4d03-8c04-d08806e40e08-kube-api-access-784dj\") pod \"maas-api-7f797f8588-t9v9s\" (UID: \"708bd367-46ef-4d03-8c04-d08806e40e08\") " pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.667152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.666967    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/708bd367-46ef-4d03-8c04-d08806e40e08-openshift-service-ca\") pod \"maas-api-7f797f8588-t9v9s\" (UID: \"708bd367-46ef-4d03-8c04-d08806e40e08\") " pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.667152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.667049    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/708bd367-46ef-4d03-8c04-d08806e40e08-maas-api-tls\") pod \"maas-api-7f797f8588-t9v9s\" (UID: \"708bd367-46ef-4d03-8c04-d08806e40e08\") " pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.667152 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.667078    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/708bd367-46ef-4d03-8c04-d08806e40e08-metrics-certs\") pod \"maas-api-7f797f8588-t9v9s\" (UID: \"708bd367-46ef-4d03-8c04-d08806e40e08\") " pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.768445 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.768399    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/708bd367-46ef-4d03-8c04-d08806e40e08-maas-api-tls\") pod \"maas-api-7f797f8588-t9v9s\" (UID: \"708bd367-46ef-4d03-8c04-d08806e40e08\") " pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.768445 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.768445    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/708bd367-46ef-4d03-8c04-d08806e40e08-metrics-certs\") pod \"maas-api-7f797f8588-t9v9s\" (UID: \"708bd367-46ef-4d03-8c04-d08806e40e08\") " pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.768755 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.768581    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-784dj\" (UniqueName: \"kubernetes.io/projected/708bd367-46ef-4d03-8c04-d08806e40e08-kube-api-access-784dj\") pod \"maas-api-7f797f8588-t9v9s\" (UID: \"708bd367-46ef-4d03-8c04-d08806e40e08\") " pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.768755 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.768615    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/708bd367-46ef-4d03-8c04-d08806e40e08-openshift-service-ca\") pod \"maas-api-7f797f8588-t9v9s\" (UID: \"708bd367-46ef-4d03-8c04-d08806e40e08\") " pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.769326 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.769291    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/708bd367-46ef-4d03-8c04-d08806e40e08-openshift-service-ca\") pod \"maas-api-7f797f8588-t9v9s\" (UID: \"708bd367-46ef-4d03-8c04-d08806e40e08\") " pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.770945 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.770924    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/708bd367-46ef-4d03-8c04-d08806e40e08-maas-api-tls\") pod \"maas-api-7f797f8588-t9v9s\" (UID: \"708bd367-46ef-4d03-8c04-d08806e40e08\") " pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.771040 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.770990    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/708bd367-46ef-4d03-8c04-d08806e40e08-metrics-certs\") pod \"maas-api-7f797f8588-t9v9s\" (UID: \"708bd367-46ef-4d03-8c04-d08806e40e08\") " pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.776903 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.776873    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-784dj\" (UniqueName: \"kubernetes.io/projected/708bd367-46ef-4d03-8c04-d08806e40e08-kube-api-access-784dj\") pod \"maas-api-7f797f8588-t9v9s\" (UID: \"708bd367-46ef-4d03-8c04-d08806e40e08\") " pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.900737 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.900698    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:54.925551 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:54.925516    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:47:55.174761 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:55.174714    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"]
Sep 08 15:47:55.177173 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:47:55.177146    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod708bd367_46ef_4d03_8c04_d08806e40e08.slice/crio-3700a4ad55cd0406da1803ab7614a0b1e5de2f82549de2411ecb4f40b3c5afac WatchSource:0}: Error finding container 3700a4ad55cd0406da1803ab7614a0b1e5de2f82549de2411ecb4f40b3c5afac: Status 404 returned error can't find the container with id 3700a4ad55cd0406da1803ab7614a0b1e5de2f82549de2411ecb4f40b3c5afac
Sep 08 15:47:55.178641 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:55.178617    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:47:55.833005 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:55.832964    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s" event={"ID":"708bd367-46ef-4d03-8c04-d08806e40e08","Type":"ContainerStarted","Data":"598cd0722892725d4208d73d0a74b78f25f2509fd7baf43d70cc98009093ba59"}
Sep 08 15:47:55.833416 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:55.833034    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s" event={"ID":"708bd367-46ef-4d03-8c04-d08806e40e08","Type":"ContainerStarted","Data":"3700a4ad55cd0406da1803ab7614a0b1e5de2f82549de2411ecb4f40b3c5afac"}
Sep 08 15:47:55.833416 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:55.833082    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:47:55.854964 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:47:55.854908    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s" podStartSLOduration=1.449824513 podStartE2EDuration="1.854890172s" totalImagesPullingTime="405.065659ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:47:55.852193352 +0000 UTC m=+2323.938143404" watchObservedRunningTime="2026-09-08 15:47:55.854890172 +0000 UTC m=+2323.940840214"
Sep 08 15:48:01.844044 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:01.843967    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f797f8588-t9v9s"
Sep 08 15:48:02.839518 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:02.839467    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:48:12.655075 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:12.655004    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:48:13.188620 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:13.188586    2575 scope.go:117] "RemoveContainer" containerID="40f10e5415f807f1c34164890e8c33cfd69f710957a5c3feefeca90dcf64458f"
Sep 08 15:48:20.545601 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:20.545565    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:48:28.738707 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:28.738667    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-s8s9s"]
Sep 08 15:48:36.568048 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:36.568017    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66db8bdb84-8j75r_ff469016-0608-4fa7-bd5b-2ffff8ca88f7/copy-manifests/0.log"
Sep 08 15:48:36.575234 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:36.575216    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66db8bdb84-8j75r_ff469016-0608-4fa7-bd5b-2ffff8ca88f7/manager/0.log"
Sep 08 15:48:37.021393 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:37.021365    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7f797f8588-t9v9s_708bd367-46ef-4d03-8c04-d08806e40e08/maas-api/0.log"
Sep 08 15:48:38.796509 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:38.796473    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-s8s9s_1553df11-ed6d-436c-a39f-007c9f4b5b59/limitador/0.log"
Sep 08 15:48:39.588268 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:39.588239    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5674d6d4b5-jhp44_2d1c1fee-f119-466d-96c6-04503d2971c4/kube-auth-proxy/0.log"
Sep 08 15:48:39.708432 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:39.702518    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qwsx4_544ecacb-8bcf-466f-b18e-f21b5509e587/istio-proxy/1.log"
Sep 08 15:48:40.510951 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:40.510920    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-ndt97_986d3075-ed53-4af2-a506-91e614440814/main/0.log"
Sep 08 15:48:40.961851 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:40.961820    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-glqq4_6dfd08ce-a73f-4134-b3b7-dc18e5a8f878/main/0.log"
Sep 08 15:48:41.070349 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:41.070320    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-ndq46_a8c8d6a0-cae9-4f40-8159-bf2d571d1df9/main/0.log"
Sep 08 15:48:46.628878 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:46.628846    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:48:46.695939 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:46.695909    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log" limit=1024
Sep 08 15:48:51.026306 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:51.026279    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lbq79_18237a47-8e2c-42f5-b960-f56188420981/global-pull-secret-syncer/0.log"
Sep 08 15:48:51.126473 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:51.126448    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8fh7x_d64afe66-e605-40e6-a64f-11dd8f2165cf/konnectivity-agent/0.log"
Sep 08 15:48:51.213457 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:51.213430    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-76.ec2.internal_b674e83ab0483a337814cf332a4de8c1/haproxy/0.log"
Sep 08 15:48:56.198355 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:56.198330    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-s8s9s_1553df11-ed6d-436c-a39f-007c9f4b5b59/limitador/0.log"
Sep 08 15:48:57.844731 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:57.844705    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-lqb82_003e44e4-48a6-4f23-8469-a4345edf59a5/cluster-monitoring-operator/0.log"
Sep 08 15:48:57.876082 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:57.876052    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-xw6hd_531d0205-8276-4123-be0b-cd0056cd2aef/kube-state-metrics/0.log"
Sep 08 15:48:57.899956 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:57.899935    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-xw6hd_531d0205-8276-4123-be0b-cd0056cd2aef/kube-rbac-proxy-main/0.log"
Sep 08 15:48:57.927701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:57.927681    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-xw6hd_531d0205-8276-4123-be0b-cd0056cd2aef/kube-rbac-proxy-self/0.log"
Sep 08 15:48:57.964941 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:57.964885    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-565b9cbcb7-722mx_d6a5cd3d-c334-41a7-b578-1960e91c49bb/metrics-server/0.log"
Sep 08 15:48:58.164936 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.164913    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bvj8f_98a5f71a-b85d-4ca6-ba83-2b8cb7835687/node-exporter/0.log"
Sep 08 15:48:58.188729 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.188709    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bvj8f_98a5f71a-b85d-4ca6-ba83-2b8cb7835687/kube-rbac-proxy/0.log"
Sep 08 15:48:58.215538 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.215460    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bvj8f_98a5f71a-b85d-4ca6-ba83-2b8cb7835687/init-textfile/0.log"
Sep 08 15:48:58.247484 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.247464    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-9s9nz_d0f2c8a0-259e-4e85-b78f-42be03476971/kube-rbac-proxy-main/0.log"
Sep 08 15:48:58.271246 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.271228    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-9s9nz_d0f2c8a0-259e-4e85-b78f-42be03476971/kube-rbac-proxy-self/0.log"
Sep 08 15:48:58.297174 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.297154    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-9s9nz_d0f2c8a0-259e-4e85-b78f-42be03476971/openshift-state-metrics/0.log"
Sep 08 15:48:58.359710 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.359687    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4aaf26c4-7510-495c-b25e-3d47fe66718a/prometheus/0.log"
Sep 08 15:48:58.399329 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.399311    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4aaf26c4-7510-495c-b25e-3d47fe66718a/config-reloader/0.log"
Sep 08 15:48:58.459382 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.459345    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4aaf26c4-7510-495c-b25e-3d47fe66718a/thanos-sidecar/0.log"
Sep 08 15:48:58.503267 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.503219    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4aaf26c4-7510-495c-b25e-3d47fe66718a/kube-rbac-proxy-web/0.log"
Sep 08 15:48:58.531929 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.531912    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4aaf26c4-7510-495c-b25e-3d47fe66718a/kube-rbac-proxy/0.log"
Sep 08 15:48:58.566055 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.566037    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4aaf26c4-7510-495c-b25e-3d47fe66718a/kube-rbac-proxy-thanos/0.log"
Sep 08 15:48:58.589432 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.589418    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4aaf26c4-7510-495c-b25e-3d47fe66718a/init-config-reloader/0.log"
Sep 08 15:48:58.794672 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.794612    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fcdbd7b65-dfmjt_191eb204-5730-4132-a3fc-4c3fa119f2c4/thanos-query/0.log"
Sep 08 15:48:58.822913 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.822893    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fcdbd7b65-dfmjt_191eb204-5730-4132-a3fc-4c3fa119f2c4/kube-rbac-proxy-web/0.log"
Sep 08 15:48:58.846373 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.846355    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fcdbd7b65-dfmjt_191eb204-5730-4132-a3fc-4c3fa119f2c4/kube-rbac-proxy/0.log"
Sep 08 15:48:58.868975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.868957    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fcdbd7b65-dfmjt_191eb204-5730-4132-a3fc-4c3fa119f2c4/prom-label-proxy/0.log"
Sep 08 15:48:58.893065 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.893044    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fcdbd7b65-dfmjt_191eb204-5730-4132-a3fc-4c3fa119f2c4/kube-rbac-proxy-rules/0.log"
Sep 08 15:48:58.920645 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:48:58.920623    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fcdbd7b65-dfmjt_191eb204-5730-4132-a3fc-4c3fa119f2c4/kube-rbac-proxy-metrics/0.log"
Sep 08 15:49:00.203996 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:00.203969    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-xnr24_b2eb0e27-7888-4a1c-bbc8-e7aed028d5a3/networking-console-plugin/0.log"
Sep 08 15:49:00.825924 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:00.825894    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:49:00.830324 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:00.830302    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/3.log"
Sep 08 15:49:01.969758 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:01.969722    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"]
Sep 08 15:49:01.973579 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:01.973561    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:01.976554 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:01.976525    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lskt4\"/\"kube-root-ca.crt\""
Sep 08 15:49:01.976673 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:01.976584    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lskt4\"/\"openshift-service-ca.crt\""
Sep 08 15:49:01.977791 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:01.977776    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lskt4\"/\"default-dockercfg-psrfj\""
Sep 08 15:49:01.984819 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:01.984799    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"]
Sep 08 15:49:02.049171 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.049141    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n25p\" (UniqueName: \"kubernetes.io/projected/92ece29a-109f-47da-8132-2c4a9526b6f7-kube-api-access-4n25p\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.049276 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.049207    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/92ece29a-109f-47da-8132-2c4a9526b6f7-proc\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.049276 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.049252    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92ece29a-109f-47da-8132-2c4a9526b6f7-sys\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.049369 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.049296    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92ece29a-109f-47da-8132-2c4a9526b6f7-lib-modules\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.049369 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.049323    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/92ece29a-109f-47da-8132-2c4a9526b6f7-podres\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.150133 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.150108    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92ece29a-109f-47da-8132-2c4a9526b6f7-lib-modules\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.150227 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.150145    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/92ece29a-109f-47da-8132-2c4a9526b6f7-podres\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.150227 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.150195    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n25p\" (UniqueName: \"kubernetes.io/projected/92ece29a-109f-47da-8132-2c4a9526b6f7-kube-api-access-4n25p\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.150387 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.150362    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92ece29a-109f-47da-8132-2c4a9526b6f7-lib-modules\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.150387 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.150377    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/92ece29a-109f-47da-8132-2c4a9526b6f7-proc\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.150590 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.150396    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/92ece29a-109f-47da-8132-2c4a9526b6f7-podres\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.150590 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.150432    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92ece29a-109f-47da-8132-2c4a9526b6f7-sys\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.150590 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.150464    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92ece29a-109f-47da-8132-2c4a9526b6f7-sys\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.150590 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.150435    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/92ece29a-109f-47da-8132-2c4a9526b6f7-proc\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.159892 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.159863    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n25p\" (UniqueName: \"kubernetes.io/projected/92ece29a-109f-47da-8132-2c4a9526b6f7-kube-api-access-4n25p\") pod \"perf-node-gather-daemonset-cvg5f\" (UID: \"92ece29a-109f-47da-8132-2c4a9526b6f7\") " pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.168158 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.168140    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-skc22_3e322dfd-60a8-4a4a-90c3-7ac16052fac2/volume-data-source-validator/0.log"
Sep 08 15:49:02.283206 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.283120    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:02.556108 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:02.556081    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"]
Sep 08 15:49:02.558513 ip-10-0-132-76 kubenswrapper[2575]: W0908 15:49:02.558477    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod92ece29a_109f_47da_8132_2c4a9526b6f7.slice/crio-123c78e92778f969a4377ff6b55e671dc8c3bed232793420ab7d87cb3536730a WatchSource:0}: Error finding container 123c78e92778f969a4377ff6b55e671dc8c3bed232793420ab7d87cb3536730a: Status 404 returned error can't find the container with id 123c78e92778f969a4377ff6b55e671dc8c3bed232793420ab7d87cb3536730a
Sep 08 15:49:03.151762 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:03.151723    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f" event={"ID":"92ece29a-109f-47da-8132-2c4a9526b6f7","Type":"ContainerStarted","Data":"43ff9ac23641381463ba423ce019d9ab76572f9fd53392cbab1d6bb2758857a1"}
Sep 08 15:49:03.152109 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:03.151799    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f" event={"ID":"92ece29a-109f-47da-8132-2c4a9526b6f7","Type":"ContainerStarted","Data":"123c78e92778f969a4377ff6b55e671dc8c3bed232793420ab7d87cb3536730a"}
Sep 08 15:49:03.152109 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:03.151909    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:03.170897 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:03.170851    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f" podStartSLOduration=2.170836318 podStartE2EDuration="2.170836318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:49:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:49:03.169689104 +0000 UTC m=+2391.255639147" watchObservedRunningTime="2026-09-08 15:49:03.170836318 +0000 UTC m=+2391.256786359"
Sep 08 15:49:03.381115 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:03.381089    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p4szr_f621c1ae-a9a4-4c7d-979c-adfbcafaf33a/dns/0.log"
Sep 08 15:49:03.403804 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:03.403736    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p4szr_f621c1ae-a9a4-4c7d-979c-adfbcafaf33a/kube-rbac-proxy/0.log"
Sep 08 15:49:03.486141 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:03.486114    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4ltq9_d33f854f-48e2-45ca-8289-0f5cb0817be7/dns-node-resolver/0.log"
Sep 08 15:49:04.043277 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:04.043255    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c4d6bc854-bq6mc_17a6542b-38ff-4c74-aa24-cf8f5f3845a3/registry/0.log"
Sep 08 15:49:04.051507 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:04.051453    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c4d6bc854-bq6mc_17a6542b-38ff-4c74-aa24-cf8f5f3845a3/registry/1.log"
Sep 08 15:49:04.102114 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:04.102091    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qvxxd_dea9369e-6c20-4b59-a04b-eb20d2f8cca5/node-ca/0.log"
Sep 08 15:49:05.384392 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:05.384362    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5674d6d4b5-jhp44_2d1c1fee-f119-466d-96c6-04503d2971c4/kube-auth-proxy/0.log"
Sep 08 15:49:05.414700 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:05.414672    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qwsx4_544ecacb-8bcf-466f-b18e-f21b5509e587/istio-proxy/0.log"
Sep 08 15:49:05.423524 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:05.423504    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qwsx4_544ecacb-8bcf-466f-b18e-f21b5509e587/istio-proxy/1.log"
Sep 08 15:49:06.396869 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:06.396840    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nvc47_a2e96079-8d75-4cd8-b77a-49f86c67a787/serve-healthcheck-canary/0.log"
Sep 08 15:49:06.918097 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:06.918066    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-8g8pd_65032153-eabb-470d-ac9c-0490d906994f/insights-operator/0.log"
Sep 08 15:49:06.918707 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:06.918689    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-8g8pd_65032153-eabb-470d-ac9c-0490d906994f/insights-operator/1.log"
Sep 08 15:49:06.940055 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:06.940034    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5wrcn_24d5bfe3-9e61-4007-8057-377731d84ad3/kube-rbac-proxy/0.log"
Sep 08 15:49:06.962393 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:06.962374    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5wrcn_24d5bfe3-9e61-4007-8057-377731d84ad3/exporter/0.log"
Sep 08 15:49:06.987473 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:06.987450    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5wrcn_24d5bfe3-9e61-4007-8057-377731d84ad3/extractor/0.log"
Sep 08 15:49:09.172863 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:09.172835    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lskt4/perf-node-gather-daemonset-cvg5f"
Sep 08 15:49:09.813973 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:09.813944    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66db8bdb84-8j75r_ff469016-0608-4fa7-bd5b-2ffff8ca88f7/manager/0.log"
Sep 08 15:49:09.834736 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:09.834713    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66db8bdb84-8j75r_ff469016-0608-4fa7-bd5b-2ffff8ca88f7/copy-manifests/0.log"
Sep 08 15:49:11.392398 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:11.392328    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5674d6d4b5-jhp44_2d1c1fee-f119-466d-96c6-04503d2971c4/kube-auth-proxy/0.log"
Sep 08 15:49:11.402330 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:11.402301    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7f7779588f-hkxc7_a9215463-91d7-47fa-849b-57a626693cb1/manager/0.log"
Sep 08 15:49:11.414701 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:11.414675    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qwsx4_544ecacb-8bcf-466f-b18e-f21b5509e587/istio-proxy/0.log"
Sep 08 15:49:11.424182 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:11.424161    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qwsx4_544ecacb-8bcf-466f-b18e-f21b5509e587/istio-proxy/1.log"
Sep 08 15:49:12.521283 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:12.521249    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:49:12.533091 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:12.533070    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-nljdd_3bf2739d-5bf1-43ee-a4a2-e2c75041d761/console-operator/2.log"
Sep 08 15:49:13.391738 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:13.391701    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5674d6d4b5-jhp44_2d1c1fee-f119-466d-96c6-04503d2971c4/kube-auth-proxy/0.log"
Sep 08 15:49:13.414007 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:13.413972    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qwsx4_544ecacb-8bcf-466f-b18e-f21b5509e587/istio-proxy/0.log"
Sep 08 15:49:13.425538 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:13.425515    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qwsx4_544ecacb-8bcf-466f-b18e-f21b5509e587/istio-proxy/1.log"
Sep 08 15:49:16.531115 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:16.531088    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-89stb_2d8c8142-9b59-4ac4-8f4b-96d48e3f40cd/migrator/0.log"
Sep 08 15:49:16.552412 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:16.552384    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-89stb_2d8c8142-9b59-4ac4-8f4b-96d48e3f40cd/graceful-termination/0.log"
Sep 08 15:49:16.909510 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:16.909471    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-cwfvk_742c48dd-0b32-418e-afb4-61f32954fbfd/kube-storage-version-migrator-operator/1.log"
Sep 08 15:49:16.910305 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:16.910287    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-cwfvk_742c48dd-0b32-418e-afb4-61f32954fbfd/kube-storage-version-migrator-operator/0.log"
Sep 08 15:49:18.254257 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:18.254228    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fchvk_e9b25500-9302-461f-9458-f2e4a5dfbcd7/kube-multus-additional-cni-plugins/0.log"
Sep 08 15:49:18.276618 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:18.276599    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fchvk_e9b25500-9302-461f-9458-f2e4a5dfbcd7/egress-router-binary-copy/0.log"
Sep 08 15:49:18.297209 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:18.297193    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fchvk_e9b25500-9302-461f-9458-f2e4a5dfbcd7/cni-plugins/0.log"
Sep 08 15:49:18.319424 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:18.319402    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fchvk_e9b25500-9302-461f-9458-f2e4a5dfbcd7/bond-cni-plugin/0.log"
Sep 08 15:49:18.341217 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:18.341200    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fchvk_e9b25500-9302-461f-9458-f2e4a5dfbcd7/routeoverride-cni/0.log"
Sep 08 15:49:18.361975 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:18.361957    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fchvk_e9b25500-9302-461f-9458-f2e4a5dfbcd7/whereabouts-cni-bincopy/0.log"
Sep 08 15:49:18.384109 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:18.384093    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fchvk_e9b25500-9302-461f-9458-f2e4a5dfbcd7/whereabouts-cni/0.log"
Sep 08 15:49:18.523706 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:18.523641    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zrlxq_e13f4f32-5641-4b50-8b14-e94551c552c0/kube-multus/0.log"
Sep 08 15:49:18.545549 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:18.545529    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9vwnd_4f348729-57f1-40ab-ae97-85bfefa7d80e/network-metrics-daemon/0.log"
Sep 08 15:49:18.562525 ip-10-0-132-76 kubenswrapper[2575]: I0908 15:49:18.562507    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9vwnd_4f348729-57f1-40ab-ae97-85bfefa7d80e/kube-rbac-proxy/0.log"